XR for Business

Geenee is an AR content company granting the wishes of their clients by creating new ways to market to their audiences, from interactive book covers, to hosting live concerts digitally. Geenee’s Cory Grenier and Elena de Sosa explain how those who master this new communication format today will dominate the market tomorrow.

Alan: Hey, everybody, Alan Smithson here. Today, we're speaking with Cory Grenier, the chief revenue officer, and Elena DeSosa, director of strategic partnerships at Geenee, a platform that delivers cost effective WebAR and scalable image recognition to the mobile web, powering XR experiences with no app required. We'll be talking about the power of the spatial web to connect us across time, geographies, and space to transform our workplaces and give us superpowers that drive commerce. All that and more, coming up next on the XR for Business podcast.

Without further ado, I'd like to introduce Cory and Elena to the show. Welcome.

Cory: Hi. It's good to be here, Alan.

Elena: Thank you for having us.

Alan: Thank you so much for joining me, guys. Cory, we'll start with you. You're the chief revenue officer at Geenee. I met with one of your colleagues, Luke, a few years ago. We became friends. He showed me this Geenee platform. Maybe just give us the sales pitch on what is Geenee?

Cory: Geenee is a new SAS WebAR platform, which is-- a scheme on your mind, it's like a Wix for WebAR. So before Wix or Squarespace, there were a lot of web developers, and it was really expensive to hire a developer to make a website. And then there was these companies that came along, decades after the initial Internet, and simplified that creation process through templates and intuitive interface. And that's what we've done with Geenee. And so we have years of proprietary image recognition technology and also tech IP and AR for Web browsers. And so we brought that together in a templatized form to allow anybody to create, publish, and share WebAR experiences directly to the web, without requiring an app.

Alan: You've created these, I guess, SLAM algorithms and image recognition algorithms. One of the things that Luke showed me before was the ability to track moving objects or videos. That was pretty cool.

Cory: Yeah. And so we've been leveraging that for a range of businesses across the entertainment, book publishing, and CPG brands, to promote soft drinks and so forth. And so it's really unconstrained what vertical you're in, how you can use the power of spatial computing to connect to your consumers and ultimately transact.

Alan: So are there examples that we can put in the show notes, that link directly to a live example?

Cory: Yeah, there's many that we can direct your audience to. Scholastic just released a new book in the series of The Hunger Games, and we brought the cover to life in an augmented reality experience on the Web. And at the end, you can buy the book across any channel. So we actually have all the purchase sites integrated for some of the movie releases recently. Even during the time of COVID, the studios are looking to connect with consumers in the home and you can experience the film in a 360 environment through AR and learn fan trivia, but also to make a purchase. Every time we believe that AR is a feature, it's not a product, and it's how you use that to connect directly to the business result that the client is looking for to get ROI in that ad spend.

Alan: Interesting. It's great. I was just looking at it, and if you want to try, then go to geenee.me, and then you can just search down the case studies and The Hunger Games one is there. So that's pretty cool. How does it work when a customer comes to you, they say, "Hey, I have an idea for a project, I want to promote this book, I want to put a AR on the front cover." Do you have developers you work with? Do you have developers in-house? Do they build it themselves? How does it work?

Cory: Yes, so we have a robust engineering team, primarily based out of Berlin, with a couple other engineers in Ukraine. And our CTO is here in Los Angeles, where we're headquartered. And so we bring a lot of world class PhDs to create and deploy these technologies and make them accessible to everyday people. So there is no coding required when you work with Geenee. Oftentimes it's the internal marketing teams that supply the assets. Sometimes they connect us to their advertising agency of choice. Once we have those assets, whether it's 2D or 3D, a digital avatar, a movie trailer -- it's really unconstrained to the imagination -- then we can leverage our CDN or Content Delivery Network, all of our back-end infrastructure and our front-end infrastructure to easily create, publish and share these experiences directly to the Web. And the turnaround time is usually less than two weeks.

Alan: Amazing. So where does the art come from? And I'm sorry, Elena, you're on the call as well. Let's have you introduce yourself. I'm not used to this three-way control here. Passing over the microphone, the baton to Elena. What is your role, and how did you get involved in this?

Elena: So I'm the director of partnerships at Geenee. And my background: actually, I started at Google Glass when that project was underway in 2013 and 2014. So that was my first foray into augmented reality. And I really love that experience, because I got to interface with business leaders and developers in all sorts of industries -- whether it was entertainment, or music, or education, or manufacturing -- and just hear the different ideas for how people wanted to use AR and what applications they wanted to build. Sort of that ultimate concept of what they could really do with the technology, because at that time it was really fresh and people had had a lot of exposure to it. So this was really people's first time in many cases, getting to try on Glass and getting to dream up anything that they wanted to do. And while Glass ended up being a little too ahead of its time -- and there's many reasons that you can give for why it didn't work out -- it just really got me excited about the future and what was possible. And from there, I went on agency side to work on innovation teams. So I worked with companies like Warner Brothers, and Hilton, and Paramount Pictures to find creative applications of technology to enhance their advertising campaigns. When I had this opportunity come up with Geenee, I was really excited to be on the other side of that and be that technology provider, and power those experiences that people have been talking about for so long and finally get to see those come to life. And at Geenee I worked really closely with our different clients to make sure that the things that they want to produce come out to spec. We help them think through the user experience. We help them think through designing for augmented reality, because a lot of times that's new for people. And I just make sure that nothing goes wrong. Basically, I make sure our clients are happy and I make sure that we're making magic.

Alan: So what's your favorite project that you've done so far?

Elena: That's a good question. I really loved working on The Hunger Games project, because I had been such a fan of those books and movies when they first came out. And I also really loved It 2. We worked with Warner Brothers in the UK and you could recognize all of their out-of-home advertisements to trigger a surprise selfie experience, where Pennywise from the movie It came, jumped into your camera. At first you would see a balloon float up, sort of like in the movie. And then without telling the user, we activated their front facing camera and then it ended up taking a surprise selfie that was baked into a personalized version of the trailer. And then they could save that out. And then that also linked ticket purchases and stuff like that. So that was a really, really fun one, and a cool way to just be a little bit disruptive.

Alan: I have a question on that, actually. Have you been able to show an uplift or an uptick in sales? Has this led to real conversions? Can you talk about numbers? Do you have real numbers?

Cory: We do. And what we saw with that specific campaign with It 2, is that five percent of everybody that went into that experience clicked the ticketing link we connected to Fandango. And that's pretty high. When you think about performance marketing and your advertising on social media, anything around the two percent is standard. So we more than doubled that by having a fun and entertaining, interactive and more immersive multisensory experience that ultimately led to higher transactions.

Alan: See, that's awesome. That's what people want to know, they're like "How do I make more money with this thing?" What do you see for the next 12 to 18 months? What are people going to do with these technologies? What do you hope or what are you working on, maybe that you're working on something that'll come out soon, or what do you see?

Elena: I think right now people are really excited about AR, because it's providing a bit of a stop-gap with everything shutting down with the pandemic. It's still a way to kind of deliver these entertaining experiences at home. And it's also something to do. I do think that that's really been a catalyst to just drive more interest and excitement around AR and just more engagement with it in general. And we're actually working on a project right now, where we're replicating the listening party experience. You've seen a lot of virtual reality concerts. I think there's one with The Weekends that's coming out on TikTok in the next few days, or may have already come out.

Alan: Tomorrowland sold out three times more tickets than they would have normally.

Elena: Wow. But yeah, I mean, we're working on this AR listening party, which is basically going to take that AR listening experience where everyone would gather in person and the musician would perform. And it would be really special for everybody who is able to be there in person, small crowd style thing. And we're going to try and make it equally special in augmented reality, where certain industry folks can get a preview of the album that's being released. And there's going to be special visuals that match up with that, that are going to be really cool and different from what you've seen before and just augment that listening experience a lot. So you're just not hearing that album, but you're also seeing the vision that the artist had for it.

Alan: It's very cool. So I guess this would be a combination of 2D assets and 3D assets visualized in the 3D world, is that correct?

Elena: Yeah, it's going to be 3D animated assets that we're going to pair along with the audio sound experience.

Alan: So how are you animating? Like, how are you building those experiences?

Elena: We have developers that are modeling and then animating them, and then we're we're placing them into the WebAR environment.

Alan: So let's talk more about use cases. Is this a marketing play, your marketing books and movies and other things? Do you see any use cases beyond this, in maybe medical, or do you see anything else? What's your main focus? If you were to kind of identify your ideal customer right now, what would that be?

Cory: It's currently the top tier brands and that the industry is less important, but a company that has incredible reach and scale. And so one of the CPG brands that we're in a late stage negotiation is, they have what they describe as more triggers in the world out there than any other company. And they said the only thing that rivals how many triggers they have is trees, is forests. And so there's I guess there's one hundred and ten billion impressions or brand logos out in the world produced annually by this company. And they want to--

Alan: Sounds like Coca-Cola. I can't think of any other.

Cory: Well, I can't confirm or deny it, but yeah. So that company has stated that they don't want to just play in AR. They know it's the next big platform -- as Tim Cook said just a few months ago -- since the smartphone. And they want to be the number one leader in three to five years in this marketplace. And so we're working with this client to validate some proof-of-concepts that could enable the brand to have capabilities that it just doesn't have today, and to be able to connect to its consumers at every touch point through the power of spatial computing. And there's another company, a rival to that one, which has actually a big print business. And they want to activate all of their magazines, even books, bring it all to life in WebAR. And not only the animated experience and the immersive experience, but the audio experience as well. So they sponsor a lot of top athletes, particularly in F1, and they want to have voice overs and a connection in a more multisensory experience that's not possible when you're looking at a two dimensional scene behind a pane of glass. And so it's really exciting. It's never been a greater opportunity in terms of connectivity, in terms of the power of the client devices, edge computing, as well as having billions of connected cameras. And that's why we built these technologies to bring that all together and then provide an easy and cost-effective way for brands to build these experiences on a mass scale.

Alan: Well, there you go. I don't know what else to say. So on that note, my friends, let's ask the final question that I ask, and we'll start with Elaina. What is the number one problem in the world that you want to see solved using XR technologies?

Elena: So I am, in my private life, an artist and a maker. I love DIY and that's my hobby. And I would love to see hands-free how-to tutorials that are very visual, that you can interact with for just everything in the world. But for me specifically, I want to see that with art, and just being able to learn how to create something, that maybe feels like there's a big barrier to entry, whether it be like pottery, or jewelry making, or weaving, or whatever it can be. And I think once you have a device that lets you have your hands free to create and play around at the same time, it's actually learning and be really hands on and visual with it at the same time. That's going to be amazing.

Alan: That's super cool. And Cory, same question.

Cory: I first joined at the computer space 15 years ago at a company called Lenovo, which acquired IBM Computer and later Motorola. And the reason I joined that firm was because I believed in the power of computers to connect people, and then later on joined a startup that made computerized eyewear. And that company was sold to Snapchat and became the first director at Snapchat and saw a emerging idea become a one billion dollar annual business in AR, peer-to-peer advertising with the Snapchat lenses. And so where we're at today is seeing entire industries leverage these technologies, even if they don't call it AR specifically, like Pokémon Go has now made over $3-billion. And so Geenee has focused on a vertical of advertising and marketing to nail that niche. And then once we penetrate that market and have has a sizable, dominant category leading position, then we can expand to other verticals -- as you said -- such as health care, manufacturing. It's really unconstrained on where these technologies can serve people and give them superpowers in the workplace. But one of the things that we're most excited about during this time of financial recession is the ability to leverage WebAR for charity, where there's so many important social causes that need funding. And so actually it was a Elena who came up with a concept that we've already validated, which is taking a stop sign and using our image recognition technology to recognize all stop signs. Which could be any species of stop sign in the UK, Germany. But we just took the US as a starting point. So if you go outside and you point at a stop sign, you can stop and donate to a charity. And there is multimedia that can be served up, which we've currently done, showing the video of the charity and then having the donate page at your fingertips. And we would be happy to share that after this podcast. So I think that's one way that WebAR can really make a difference and put dollars into the hands of people that need it most.

Alan: Well, that's a great use of this technology for sure. And I want to thank you guys for joining me on the podcast, and also thinking about these types of things there. Is there any last points you guys want to make before we wrap it up?

Cory: I would just say that millennials and Gen Z don't remember life before the mobile phone and by contrast, the first 20 years of our lives, Alan, you and I, when we were born and until we were 20, we weren't connected to the Internet. We didn't have all the information at our fingertips. And today, any child that is born in 2020, 20 years from now, 2040, they're not going to distinguish between extended reality or augmented reality, the mobile phone is just going to be their reality. And so the companies that adapt to that reality of spatial computing, using tools like MetaVRse, like Geenee, they'll be the ones that are positioned for market leadership in the future. Well, I'm glad you said it. Elena, carry on.

Elena: Adding on to that, I do think people are really excited about AR and that by removing the barriers that we have seen so far -- the cost, for example, or needing to code, or the general ease of creation -- and making it as easy as possible to use it, we're just going to start to see a lot more uses of it and also the possibilities of how it will be used. I'm really excited to see what new things come up, because it's been the same sort of concepts in mind for years. And I'm sure there's even more cool things out there that we haven't even explored yet.

Alan: It's amazing. Once you make a tool that allows people to build, you never know. When I was building DJ software, Mike Shinoda from Linkin Park actually took our software and built a complete modular keyboard that looks nothing like a keyboard on a touch screen.

Cory: That's awesome.

Elena: That's so cool.

Alan: Recently on our MetaVRse platform, we had somebody from Oracle -- Sikaar [Keita] -- he built an IoT sensor connected to the Web3D, and was able to control it real time. So you never know what people are going to invent when you give them tools that are kind of open and unlocked. So I'm really excited about what we can see coming out of the Geenee camp in the next little bit. Thank you so much for joining me, guys. And thank you, everyone, for listening. This is the XR for Business podcast. I would ask you to subscribe. If you're listening on the podcast, hit Subscribe. And you can visit xrforbusiness.io and sign up there, and we'll make sure that your inbox always has a new, fresh episode. Again, Cory, Elena; thank you, guys.

Elena: Thank you.

Cory: Thank you.

Developing a pilot or a proof of concept is among the first steps to introducing XR into your industry, but that’s only going to provide so much ROI unless you can fully implement that idea. Dave Beck from Foundry 45 discusses how to make that leap.

Alan: Hey, everyone, Alan Smithson here. Today, we're speaking with Dave Beck, founder and managing partner at Foundry 45, an immersive technology company that develops enterprise level virtual reality training experiences. They've created over 250 experiences for notable clients such as AT&T, Coca-Cola, Delta, and UPS. We're going to be discussing going from PoCs, pilots, and case studies to full scale deployments. All that and more, on the XR for Business podcast.

Dave, welcome to the show.

Dave: Hey, Alan, thanks so much for having me on here.

Alan: It's my absolute pleasure. I'm super excited. You guys have been doing so much work in the VR training space. First of all, let's just talk about, what is Foundry 45? How did you get into this? And we'll kick it off from there.

Dave: I guess first off, it's nice to chat with you again. I went back and checked my email, and it looks like the first time you and I talked was way back in 2016. So a lot's happened in that time, hasn't it? So, OK, we put VR to work by creating virtual reality training experiences for enterprise partners. And we specialize in industrial-- think hard skills type training. I've actually been working in immersive technology for almost a decade now. Initially it was in augmented reality, which was something that we added on the side for a SAS product we built, that was actually our main business during that time. And we did a lot of stuff where you would hold up your phone or a piece of industrial equipment, and it would tell you where to wrench on it or how to change the filter, things like that. It was cool technology, but we pretty quickly realized that no one was going to hold a phone or an iPad over a piece of industrial equipment on an oil rig. They weren't going to set it down and start wrenching on something, and then pick it up with greasy hands. [chuckles] So what we wanted to do was hands-free AR, but the technology just wasn't there. We exited that company in 2014, and we were trying to figure out what we wanted to do when we grow up.

Alan: You exited your company in 2014. Most people didn't even know what this technology was when you guys were exiting your first one and getting into the second. Think about that.

Dave: Yeah. I mean, we were trying to figure out what we wanted to be when we grew up. And one of my co-founders bought an early innovators edition of Samsung Gear VR. Do you remember that one?

Alan: I have the one with a solid strap on top.

Dave: Yeah. Did you strap the Android phone into the headset?

Alan: Exactly.

Dave: Yeah. And you could use that camera on the back of the phone as a pass-through.

Alan: Yes. Well, not very well, but yes, you could.

Dave: Well, yeah, it's funny. You know where this is going, right? Because we wanted to use it for hands-free AR, but it didn't work at all.

Alan: Not without making people very sick.

Dave: Yeah, the processor wasn't good enough. It was super laggy, which kind of made it nauseating. So that wasn't going to work. But wow, VR was awesome. That's when we made the decision to start down our current path.

Alan: And that was before ARCore and ARKit. So planer tracking really wasn't a thing. Slim mapping, it was not that easy to do.

Dave: Yeah. I mean, just being able to do something on a 2D plane was one thing. But yeah, if you wanted to actually render something in three dimensions on top of a physical product, it was super hard.

Alan: We did one of the very first WebAR projects in the world, and it sucked. Oh my god. It was so hard to do. We got it done, but man! There was no tools, there was no nothing, you're just guessing at things.

Dave: You make it up as you go.

Alan: Indeed. Carry on, Dave, sorry.

Dave: Yeah. I was just saying, that was basically how we started down our current path. We had spent most of her time in AR, but we were blown away with this new VR device. And the toolchain for developing in virtual reality was -- is -- very similar to augmented reality. And just kind of never looked back.

Alan: So, OK, now you're doing training with glasses. The headsets have come a long way. We have tetherless, wireless headsets. On the other end of the spectrum, we've got the Varjo headset that's got human eye resolution. Where do you find your customers are kind of gravitating towards? Are they looking for photo-real or does it matter to them? Is it just about the experience in training? What is your experience in that?

Dave: I think it's all about the experience. Different headsets work for different environments. We have played around the Varjo but have not actually developed anything for commercial use on it, because the people that we work with just have kind of a hard time stomaching the cost. It's amazing, it's pretty. I think that's where everything's going. But today, the biggest thing, the biggest trend that we see is moving from the PC based platforms -- the HTC Vives and the Oculus Rifts -- to the Android based platforms like Quest and Vive Focus Plus. Pico Neo 2 is another new one that that's just come on recently, it's pretty interesting.

Alan: I read SkarredGhost's -- Anthony Vitillo's -- review on it. It seems amazing, actually.

Dave: Yeah. We got a chance to play around with one of the early beta editions. Right now, the big challenge in that space is that the best quality headset that's out there is the Oculus Quest. Unfortunately, the terms and conditions of Oculus/Facebook are such that it doesn't play nice with the enterprise IT infrastructures that we work with.

Alan: And that's a massive market opportunity for Pico.

Dave: For sure. It definitely is.

Alan: I want to focus this this question, because I want to get right to the meat and potatoes of this. The headsets are out there. You guys are making training. What types of training? You said physical things, where you're training on maybe a rig. Where is the most value being created and how do you scale that?

Dave: Where we find the most value is really on the procedural training side. That's training where there's only one right way to do things. So as I was saying before, think hard skills, things where there's a standard operating procedure. There's a lot of great VR training experiences for soft skills, but at heart Foundry 45 is a company of engineers and scientists. So we're good at taking complex training procedures and realizing them in VR. And I think most importantly, the benefits are measurable in that space, so we can actually demonstrate an ROI for our clients there.

Alan: So where is the ROI? If I'm an oil and gas company and I say, "Hey, look, we got all sorts of training. Nobody can train properly, COVID times. Where do we start?"

Dave: For us, there's probably three main areas where we most often find the ROI. The first is quicker time to mastery. So we're currently working with a Fortune 100 logistics company that's actually cutting their training time and almost half, by using virtual reality. And that's gigantic savings. The second one is less equipment downtime. A lot of areas where we work, process line downtime is a huge cost. So training operators or maintenance personnel virtually, that saves a ton of money. And then the last one is lowering costs on travel to training environments.

Alan: Well, now you can't even travel.

Dave: [laughs]

Alan: So lowering costs... none of your people can travel, but that means they also can't train.

Dave: Yeah, right. I mean, you want to train your US-based team on your new production line, but it's being produced in Germany. Yeah, you can't get over there, but now you can do it in VR.

Alan: So are you seeing [that] most of these are kind of bespoke to the individual customers? And if so, how does it scale in terms of how does a customer then take maybe a base scenario and start to make changes themselves? Because if you have a training department, they don't want to be coming to a third party for every single thing. So how does that work?

Dave: I think we're pretty much in the early stages of that. There's a lot of platforms out there that are purporting to be the WordPress of VR, for example.

Alan: Is there anything you've seen that-- the only one I saw so far is Enduvo, actually. It allowed you to import some videos and 3D models. It was very intuitive, but not at the level of what you're talking about, I think

Dave: Most of what we've seen are things that are more on the 360 video side than the immersive VR space, which is where we exclusively work. I mean, there are a few that are in early stage right now, that are promising. We productize our offering by having a kind of standard toolkit that we work from. But today it's still a relationship where if we work with somebody and they want to hit pause and move forward, then they're going to have the intellectual property that we've developed for them as part of the process, to be able to actually work on as well. But having an instructional designer at a Fortune 100 being able to create their own content, in my opinion, I don't think we're quite there yet.

Alan: Ok, how long do you figure? Because the tools are getting better day by day, like it's just leaps and bounds. So based on the stuff that you're working on, because you guys know what it takes to make this.

Dave: I guess there's two ends of the spectrum. So what's good enough to actually get value from? What's the minimum viable training product, if you will? And if there's something there that's-- the things that I see that are on the more generic end of that spectrum are typically going to be opportunities that are ones that get stamped out that are the same -- could be diversity inclusion training, or slip and fall training, or something like that -- where the same piece can be sold over and over. And we do some work with cleanroom training that can actually be sold over and over as well. Those things, that's kind of the first step into that before being able to have somebody, I think, actually be able to develop their own.

Alan: It's mainly about making some content that can be reused by multiple entities, maybe just change the logo on the wall. And it's the same kind of-- so almost like WHMIS training, or chemical training, standard fire safety training in places. I actually tried a weird thing at a conference once that it's like-- if you ever tried the one where you kind of walk and your feet are slippery and you kind of walk on a disk?

Dave: Oh, like an omnidirectional--

Alan: Yeah, like an omnidirectional [treadmill].

Dave: There's actually one in our building. Not that I've been there recently but...

Alan: Oh, this was crazy. So I was walking, and as I'm walking -- because I was harnessed in -- and it was pivoting up and it was a fire safety training. So I was walking around in VR but I was using my feet to walk, which was a strange thing, because you're kind of hanging from this thing and your feet are kind of dangling below you. It made me so sick. [laughs] I was like, "OK, we can't use this."

Dave: I mean, the one that we have is the Omni by Virtuix.

Alan: Yeah, yeah, the Virtuix.

Dave: Yeah. I mean, for me, I guess in general, when we develop experiences for people, there's always questions around can we use haptic gloves, or can we use a special vest, or all these kind of peripheral things, or create something that's really specific to a chair or something.

Alan: A level of-- it's exponentially more difficult, each thing you add, for now. until they all become--

Dave: We always encourage people to, until those things are more baked, I think there's very good use cases for them, so there's very specific use cases for them, but typically when we're developing things, we want the headset to be applicable to more than one specific operative. So, for example, Delta Airlines could buy a deicing VR simulator. But it's a $100,000 piece of equipment that only does deicing and sits in one place.

Alan: I've actually been in that boring simulator.

Dave: Which is really cool.

Alan: It's super cool. But you're like, "Wow, this job sucks." Because it just goes very slow. It's exactly what the machine does. And you can't have this thing flying around.

Dave: Yeah. And so for 1/100 of the cost of the hardware, you won't have the same exact feel. Your muscle memory may not be exact for some of the pieces. We do training on light rail. And there's a throttle, and we spent a long time talking about, well, you're not going to get the feel of the throttle, or in industrial you're not going to get the torque of the wrench.

Alan: But does it matter?

Dave: Right, exactly. The idea is that you need to get these steps, you need to do these steps in order, and you need to understand them. If the point is that we're trying to teach somebody how to turn a wrench, that's just not-- it's not good for that right now.

Alan: Understood. I've actually tried it. I've tried weighted controllers and I've tried the haptics gloves. And it does add a whole new dimension to things, but not ready for prime time. I agree with you on that, 100 percent. So you're saying then the standalone headsets is kind of what's driving this mass adoption, and when I say mass adoption, I mean within the enterprise, because the whole idea is that you've been building lots of PoCs and pilots and building on a body of knowledge, and now you're executing this at scale. So what is the kind of feedback, can you talk about any specific things where this company saved X amount?

Dave: There's a little bit of a challenge today, because there's so many different-- there's a lot of internal data that shows huge returns of investment. But the companies, most of the large companies we work with are very hesitant to share their data externally. They talk about qualitative things, but not actually hard numbers. That's been a challenge for us, because that question comes up all the time. So what we've done is we partnered with a local university to create our own data. It's slow going, because of all the pandemic concerns. But we do-- we actually started on it in the spring and then it's been pushed off. So we're hoping to have some good data by that we can share before end of year. So there's kind of more to come there.

Alan: That's great. It's one of the questions that we get asked most. And it's the same answer. People don't want to share exact numbers and it's interesting, because I've been able to pull out a lot of data out of people on this podcast, they've been gracious enough to share it. Mohamed Rajani talking about returns at Macey's and giving specific numbers, and Jonathan Moss at Sprint talking about specific costs and numbers. When people are able to share these, that's when it really helps the industry a lot. So I'm glad to see what you're doing with the university.

Dave: Yeah, I do have one kind of piece of anec-data that was interesting. Just in the last week, I was at a senior leadership meeting at a Fortune 100, and they're literally talking about the effect on their share price that the VR training work that we're doing could potentially have. If that gives you a sense of how important this is to some folks.

Alan: People are looking out like, what are you doing to be an industry leader now? What are you doing? Are you-- why aren't you using the kick-ass technologies that exist? I think there's a definite shift in mindset around this, and the fact that they're thinking about the link between deploying this technology and their share price, that's very interesting.

Dave: Yeah. And I mean, if you want a specific example from a different company, one that we've been working with for a while is Delta Airlines, and they have this huge challenge to train the people that are on the ramp, which is the area underneath the plane, to get a plane in and out quickly and safely. They can't train on an active jetway. They can't take a plane out of service, and they can't even get employees badged by the FAA for ever. So their solution's VR training and the results after putting thousands of employees through it are-- the less concrete one is just the employee satisfaction piece, where they're actually saying they want to do more training, which never happens. But then--

Alan: Never, ever. No one ever says, "Please give me some more phone training."

Dave: Yeah. "Take me out of my actual job to do training." The cost of travel are way down and all these other pieces actually get them excited to do a lot more. And those are the things that get us excited.

Alan: How ironic that an airline would think about the cost of travel reduction?

Dave: I know! But It's a huge, I mean--

Alan: But it's such a universal thing. People don't want to pay to fly people around the country to learn something they can do with a $500 headset.

Dave: Right. Somebody's got to pay for that seat, right?

Alan: Exactly. So Dave, you've done all these hundreds and hundreds of experiences, you're starting to kind of see the ones that repeat, you're able to resell those. What's next? Do you build like a CMS, like a central repository of these experiences and say for $100 a month per employee, you can have access to this? How do you then scale this?

Dave: Yeah. So for us, the most interesting conversation that we're having right now is all around what's beyond the proof of concept. How do you actually take VR training and get it out throughout the enterprise? And we usually talk about things in kind of two steps. The first one is to create your experience. And we've kind of already talked a little bit about these types of things. But start with your goal in mind. Stay really close to the development process. Get in the headset as much as possible. And then pilot, you have to pilot your content, because there's bound to be improvements. And once you put it in front of the intended audience, there's going to be changes, you've got to budget time for that. So that's just kind of the first step. The second step after you've-- you like what you have, you've socialized it around the business, you're getting buy-in from interested parties. All those are really important. But more important is that how do you actually get a return on it? The next step is to deploy it. And you can either take that content to new plants or locations, get more people involved. You can create more content as you're honing in on where you're getting the most bang for your buck. For example, with the Delta thing, they started with plane inspections and then quickly followed with other related activities. People that hold up the flashlights on, that's called marshalling. That's a great one. But then, yeah--

Alan: I want to try that!

Dave: It's amazing.

Alan: Can I try that?

Dave: Yeah, definitely. When next time you come down to Atlanta. So, yeah, the next step that is very important is to actually integrate it directly with your internal IT systems. A lot of people have an LMS or a Learning Management System. Basically what we'll do is we'll have the VR content live inside there in the same way that you would have your 2D video content or your online skills assessments. And that's super important, because you don't want to have to keep track of your VR training in a file cabinet off to the side or something, and you don't want to have to have different logins for your employees. So for the VR training work that we do, we'll tie it in so that everything goes through a Cornerstone or Saba or SuccessFactors or something.

Alan: That really completes the circle. How do you deal with device management? That's another question that's come up a few times on this podcast. You figure that's going to be built into the system kind of thing, or is that a third party?

Dave: So we actually developed our own kind of virtual reality management system to be able to do that, to stand up multiple instances of the same experience at different locations around the world. Ultimately, the way that I believe that should be managed is just there's plenty of good MDM solutions out there, and there's a lot of people that are actually working on it right now. And we're working with some of them directly. I think, especially as we go to more Android based mobile platforms, having a solution out there that is a great way to keep track of that is definitely important.

Alan: And I think the other thing is going to be revisioning or keeping track of who changes what as it goes through.

Dave: Yeah, definitely.

Alan: Today you're building your own toolset and then you're going, "Oh, I guess if we need it, everybody is going to need it." What's next on the radar as far as out there R&D stuff? What are the things that get you going, like "Man, if we can just solve this problem?"

Dave: Well--

Alan: You don't have to answer.

Dave: No, no! What's interesting for us, I think, on the technical-- on the hardware side, everything is getting smaller, cheaper, better, faster, the typical technology curve. But with experiences, I think we're really just scraping the surface. People are just figuring out what's working, what's not working. And a lot of that is trial and error. So obviously, working with somebody who's been there, done that is helpful. Mobile, for me, that's where it's at right now. It's just so much easier to deploy. It's less expensive. And just put it on and go.

Alan: Indeed, it has the potential for massive scale. What problem in the world do you want to see solved using XR technology and why?

Dave: Overall, I guess for a second, let me mention kind of the the macro environment that we're in and the effect that's had on everyone. I mean, it's been a crazy few months here. Training is more important now than ever. And that's not just with new employees. Reskilling and upskilling are super important, too. We have one client who's transitioning from conventional facilities to more automated ones. They have a very strong, very tenured staff, and they don't want to just push them aside, so they're using VR training to help upskill them for new roles and new environments. And I think for us, that's the most exciting thing right now, being able to impact people's success in their job, workforce wellness for them. That's what gets us really excited. That's what gets us up every day and inspires us to do more.

Alan: Amazing. Thanks for listening. This has been the XR for Business podcast with your host, Alan Smithson and today's guest, Dave Beck. If you want to learn more about the great work Dave and his team are doing, visit foundry45.com.

Of all the jobs that are difficult to train for, surgery is especially challenging, what with needing a body and all. But our guest Aravind Upadhyaya is working to make surgical training virtual, with the help of XR technologies.

Alan: Hey, everyone, Alan Smithson, host of the XR for Business podcast. Today, we're speaking with Aravind Upadhyaya, co-founder and CEO of 8chili, an Oakland, California based startup, bringing the dream of telesurgery to the real world. Today, we're going to talk about how virtual reality is improving the outlook for remote surgeries globally. All that and more, coming up next on the XR for Business podcast.

I want to welcome Aravind, thank you so much for joining me on the call.

Aravind: Thanks, Alan. It's a real pleasure to be on your podcast, and to get to know you, as well.

Alan: It's such an honor. You guys are working on something that is a true game changer for surgeries and remote telemedicine. Maybe just tell us what you're doing and the problem you're solving.

Aravind: Definitely. So, just to do a quick intro: I'm a technologist, so I'm an electronics engineer. I've spent the last two decades leading R&D projects in mixed reality, computer vision, Internet of Things, and robotics at one of the biggest, largest conglomerates, the Tata Group. And in 2016, I had my Tony Stark moment when I had the Hololens. It kind of felt like I could hold the power of x-rays in my eyes. And that's when the journey started for 8chili. So we've been working on this technology with surgeons for the last two years, kind of in stealth mode. And this year, GPM, we finally kicked it off. In a nutshell, we wanted to reimagine surgeries with mixed reality, by building a remote collaboration platform. I want to start with this: 60 percent of new residents universally are not confident to perform core procedures. And just about 3 percent of the surgeons globally have access to high quality continuous training. And why is that? And that's because we have a very legacy training system. So, there's a great adage that goes like, "I hear and I forget. I see. And I remember. I do and I understand." And that's why we are building NAVIX AIR, because experience cannot be explained. So you have to experience to really get to be an expert. And NAVIX AIR allows residents to experience what a surgeon does in a surgery. So this, what we want to do, is take the platform to a very immersive collaborative experience. Now, NAVIX AIR is a platform that allows residents to transform into the surgeons eyes and follow their steps simultaneously in the virtual world, without disturbing the surgeon. So let's say you can fail and repeat as many times as you want, even post the live surgery. And the big difference is being able to see what the surgeon sees brings this feel of reality into the residence. And that's what is missing in the cadavers or the existing VR simulators. Because what happens is no surgery goes smoothly or perfectly. There are complications, there are surprises that happen, like another surgeons encounter. And these cannot be captured. So it's the surgeons experience that come into play when something is not going as they expect it to go. And this feel of how to react in such a situation-- let's say, what does a surgeon do? What is the communication that they used to talk to the nurse, the anesthesiologist, or the other juniors who are helping the surgeon? Or what kind of technique? Like, if there is a bleed which was unexpected, how do they go about tackling that? So all these things are scenarios that will be happening in the real world. So there's no substitute for it.

Alan: That makes a lot of sense. So when a surgeon is operating, when these things pop up, it's almost impossible to train for every scenario, but you can record it.

Aravind: Absolutely. So we allow them to log in,, where it's saved as a portal. And if you're a surgeon, Alan, let's say you put on our headband and it's nonintrusive, so it allows you to do your standard procedure the way you've been doing it all this time. And I, as a resident, can then see what your eyes see in full 3D. We transfer the entire 3D scene that you're seeing, in close to real time. Let's say you have a half a second delay right now, which we are trying to reduce it to like almost near zero latency. But even with the half a second delay, it's unnoticeable for a junior. And then I can actually see what you are doing and repeat the same steps that you are doing simultaneously with the virtual tools I have.

Alan: That's incredible. So over the course of a surgical resident's time, how much time would they actually get normally to stand over and watch surgeries?

Aravind: So we have a proxy research that goes like, 40 percent of the programs right now report fragmented training, and due to restrictions that impact skill acquisition. So that's one of the leading issues, as in why the residents are not confident in performing core procedures. So they would like to put in-- every resident start doing practical trials by the second year and -- let's say if you take neurosurgery, for example -- they do five years of residency and then two years of specialization in neurosurgery. So they have to put in at least more than 100 to 200 cases, 200 cases is the minimum that they would need to become an expert in a specific neurosurgical speciality. And that's to less, that's the minimum. But with our technology, we could enable them to learn not just from the surgeons that are in the university or in the teaching hospital, but they could to learn from expert surgeons all over the world, and any time they want to, as well.

Alan: If you think about it, we put pilots through thousands of hours of practice before we let them fly a plane with a bunch of people in it. But surgeons, it's in the hundreds of hours, because of complexity. You're allowing surgeons to have the same kind of level of training as pilots. I think I want the surgeon who's had thousands of hours of training, versus hundreds. That's just me personally.

Aravind: Absolutely, absolutely. I mean, this industry is so neglected, like the surgical training industry. And we have great tools for everything else, like navigation, and robotics, and everything. But if you look at training, it's still so archaic, you have-- if you stand in the operating room, a resident stands in the operating room, there is really no space next to the surgeon, because it's a life case. And the surgeon is in charge, so he's the captain. And there's a nurse who's next to the surgeon, and then there are tools. So there is literally no space. And the surgeon is not going to be like, "Hey, you know what, come and stand and see how I see things." No, it doesn't happen. So it's part of learning most of the time when they go to cadavers. Again, you don't really get to have the experience of the variety. That's the use case that you're trying to target. When we do the more surgical training -- and especially in life cases -- we allow our platform to be tested rigorously, and that will allow us to then port it to the bigger concept of remote surgeries itself.

Alan: So the ultimate goal is remote surgeries altogether.

Aravind: Yeah, absolutely.

Alan: This is where the telcos go, "Ooh, 5G! We need 5G in there, to reduce the latency!" Because latency would be the biggest problem with remote surgeries that I can see. [chuckles] A couple of milliseconds difference, and you're a millimeter to the left or a millimeter to the right.

Aravind: Absolutely. I mean, it's our biggest bottleneck, I would say, to realize the dream of remote surgeries. And I think probably for any other company who's working on remote collaboration, I think this is the biggest bottleneck right now. Everything else could be worked around, but not this.

Alan: Well, the good thing is there's a lot of people working on the space, so let me ask you, with what you're working on now, where in the life cycle of the product are you guys now? Are you in trials? Are you in market? Where are you now?

Aravind: So currently we are in trials. So we have used on Phantoms, we have worked with Phantom so far, because a part of our surgical training also includes surgical navigation, I should say. I would share a video, post this cast, that you would see what I'm really talking about. So this is what happens with the existing VR simulators, you're able to use the tools that are available -- for example, orthopedic surgery, like this total knee replacement -- then you have all the tools, you have on the assets and everything. And then you go about performing step by step what a surgeon usually does. Now, in our case, because we are doing a live transmission, so you're able to see what the surgeon is doing. Most of the patient is draped, so you're able to see what area the surgeon is working on. All good. But that also means the part wherein the surgeon has access to, let's say, in his guidance tools, we won't be able to show that on a video, because if the surgeon is looking at a screen and is doing navigation, that's not going to be really that effective for the resident. So what we do is, along with-- there's one extra step before starting the surgery, the surgeon has to do a call registration, basically a process whereby the preoperative scans of the patient are registered with the patient organ. And what this is allowing us to do is the surgeon won't be using it, the one who is performing the surgery. He's going to it in the conventional way -- they're going to do the surgery. But we going to use that data to superimpose these virtual assets, like what others call it. In our case, basically, let's say it's a neurosurgery, and if I have a scan -- CT scan -- I can extract the skull of the patient. I can extract the brain of the patient from the MRI. I can extract, let's say the grey matter, white matter and the DDI fibers, the tract of the visualization, all of this data which can be superimposed onto the patient. So even though the resident is seeing whatever the surgeon is seeing, he has this extra powerful feature that he can now instantly overlay onto that patient in the super accuracy. And then what happens is, he has control, so they could use the opacity and then see in detail anatomy, like what is the approach the surgeon is following? What nerves the surgeon is avoiding? That's the kind of detail that we are talking about. So we create like virtual assets on the fly by taking the patient scans. So we've done that part, we've tried that surgeons. We have deployment coming up with our adviser in immersive, Dr. Anthony Avelino. He's the provost at University of Michigan Hospital. So we're getting this deployed. The COVID is delaying stuff for us, but I think it is still on track to deploy by October.

Alan: It's amazing that you said University of Michigan, because there's a lot of work going on at the University of Michigan. Jeremy Nelson is kind of spearheading the university's mixed reality center. So they've got over 20 Hololenses there. They've been working in mixed reality for a few years now. I know everybody's really excited about that. We actually took a tour of their virtual reality lab. And it's interesting how a university took on this idea of mixed reality and then did a challenge to all the different divisions of the school and said, "Hey, how do you think you could use this technology? Come up with some ideas and we'll build it for you." That is a really interesting use case, and the fact that you guys are working with them is wonderful. So I am assuming you're going to be on Jeremy's podcast at sometime soon.

Aravind: [laughs] Well, I hope so, yeah.

Alan: You're speaking to an audience here on this podcast. What is something that you would want to tell them to check out? Who are you looking for, as far as customers, and how do they get in touch with you?

Aravind: They can get in touch with me via my email at [email protected]. We have a dedicated team that looks at all emails and get back. So we're happy to do that.

Alan: Just so everybody knows, it's 8chili.com.

Aravind: Yeah. Thank you.

Alan: Where did you where did you come up with a name? I got to ask.

Aravind: [laughs] Thanks, I was waiting for you to ask that question. [laughs] So we were looking for a name that would not have a conflict of interest with any other company that is doing augmentation or something like that. So we failed for the first 24 hours. And then the team was hungry. We ordered like super hot Indian spicy pizzas. And out of joke I was like, "Hey, you know what? We are trying to spice up this navigation industry, right? So we should call ourselves the Chili Company." So the team goes like, "That's awesome! We should go for it!" [laughs] So 8 is basically Infinity, and we a have more Marvel inspired team, so our logo is coming from Deadpool, we wanted it to be "two eyes" kind of thing. So that's the whole story with the name.

Alan: It's one of those things that you got to ask. First of all, is there anything else you think people should know about 8chili before we move on?

Aravind: Yes. Anyone who would contact us, we would like to show them a demo of how the concept looks and we would be ready for deployment by October. And we would be very happy to come over to any location in the world, where we could fly -- if there's no COVID restrictions -- or virtually also we could still show them the demo. And I suggest everyone to at least get in touch with us to see the demos, because it's really going to change the perspective of surgical training. Anyone who want collaboration.

Alan: So how can people see the demo if they don't have a device? So can they see it if they have a Quest or...?

Aravind: Yeah, we could-- we have a beacon host. We are actually hosting an app on the Oculus Quest. So if they have a VR device, they could likely go and experience it.

Alan: Great.

Aravind: Yeah. So we are trying to do a closed demo, rather than hosting it. But we are happy to do that, so that if customers want to experience what it feels like being in a 3D environment, then we're very happy to set it up for them.

Alan: So I have to warn people, if you're squeamish, am I going to be looking into people's innards?

Aravind: Yes, the demo will be-- mostly we will do it with a phantom, so that they don't get that freaky about it. So we have a phantom that we ordered from Amazon, it's a skull. If they are able to see a skull, they should be OK. But real surgeons, yes, there's a lot of blood. If you get dizzy, nausea, and stuff like that, yeah. [chuckles]

Alan: I think this is also a great way for young people to try early on. Maybe this is a good way to get young people excited about medicine, and alternatively people that thought they were excited about medicine, when they try this, maybe go, "OK, that's not for me. I don't like that." So I think it's a great way to encourage young people to explore medicine as well and see the future of medicine in a way that is fairly scalable.

Aravind: It's a super advice, I would say. I'll take that up, as one of the things that we would like to do. Because I had not thought of that, inspiring the next generation to become surgeons. But, yeah, I think it's a very good space to look at. Maybe make it a bit simpler, so that they could still experience the 3D perspective and see that this is a career to explore. That's a good space. Thanks, Alan.

Alan: This is one of the things that VR and AR do really well, is that giving people the ability to try a career, before they commit their entire lives to it. One of my friends has a company called Career Labs, and what they do is they create VR experiences, but they're short, they're maybe 15 minutes. And you get to try driving an excavator, or you get to try driving a dump truck, or you get to try driving a mining vehicle. And so you get to try these different things. Mostly his are in trades, but imagine having that and then other people can try surgery. Imagine having an entire collection of things that students could try, viscerally. Get in their VR and try, experience. Because we do need people in trades. We do need people to go into surgery. We need people to go into these. And how do you inspire people? What better way, than to put them into the experience? And I think it's a really wonderful thing that you guys have created. And there's a whole other market, just inspiring the next generation.

Aravind: Absolutely. One of the use cases that we are working on also is remote collaboration, really helps see space and defense. So imagine let's say, you are sending someone to space. And if there is a complication, it's so difficult to perform-- you can only do first aid, but there can be times when a smaller intervention is definitely important and saves lives. So we could use our technology. And that's what we're trying to work on, to make it more easy for someone on the side of doing the surgery.

Alan: I would love to see how you guys figure out how to overcome the two second latency between something done on Earth, and between the time it happens on the moon. There's a two second round trip latency. Two seconds! Two thousand milliseconds! [laughs]

Aravind: I know. Right now our latency...rather than recreating the entire scene with textures and all of those things-- you know the point cloud, so what we do is we take the complete point cloud and transfer the point cloud over edge protocols. And that allows us to really keep the RGDD data as well.

Alan: Well, that data is easy. It's when you lay textures over that it gets really complicated. So you guys are doing the right thing.

Aravind: Absolutely.

Alan: A lot of people in 3D are missing that; that if you have the 3D objects, you can-- first of all, you can depolygonate, you can take out a lot of polygons, make them smaller by simple compression.

Aravind: Yeah.

Alan: But if you don't layer them with textures, they end up becoming easy to deal with. It's when you lay on textures and transparencies, then it gets really complicated and that's when your devices start to bog down a bit. So you guys are doing it right.

Aravind: Rigging and meshing takes a lot of time and effort. So even if you move these processes to boot-up, it'll still give us a delay. So we chose to directly work only the point cloud. And our mission is basically about taking the point cloud and making it look realistic as soon as possible. [laughs]

Alan: Yeah, it's all about a bit of trickery, a little magic.

Aravind: It is, it is. [laughs]

Alan: Aravind, here's my final question. What is the one problem in the world that you would like to see solved using XR technologies?

Aravind: I would like a general practitioner or someone with general surgery experience to be able to perform a surgery with collaboration from experts anywhere in the world, and deliver that surgical care irrespective of the location. Tier three cities, tier two cities. We want everyone to get that care, we want it to be democratic and universal. So that's the problem that we want to like we saw.

Alan: Well, that sounds like a pretty damn good problem to solve, my friend. And I wish you all the best with that.

Aravind: Thank you, Alan. And we look forward to keeping you posted as and when we move closer to our mission.

Alan: I'm looking forward to it. And for those of you listening, you can subscribe to this channel so you don't miss any of it. It's xrforbusiness.io. You can subscribe on all of the channels. And I want to say, Aravind, thank you so, so much. Aravind Upadhyaya from 8chili, you can find them at 8chili.com. And thank you so much for listening to the XR for Business podcast with your host, myself, Alan Smithson. And we'll see you on the next episode. Thanks, everyone.

Aravind: Thanks, Alan.

Happy Finish’s CEO Daniel Cheetham’s XR bread and butter was virtual experiences and LBEs, until the COVID-19 pandemic forced people to stay indoors. Now, he explains, he’s exploring the power VR has to help enterprise and the environment.

Alan: Hey, everyone, Alan Smithson here. Today, we're speaking with Daniel Cheetham, CEO of Happy Finish, a creative technology and content firm based in London, UK. They've done XR experiences for Ford, Exxon Mobil, and many more. We will learn today how they're using digital twins to add real business value. All that and more, coming up next on the XR for Business podcast.

Daniel, welcome to the show, my friend.

Daniel: Thanks for having me.

Alan: It's my absolute pleasure. I've been looking forward to having you on the show for so long. It's really exciting, the stuff you guys have done. You've done everything from putting people from the real world into VR on a bicycle, to help give them a sense of what it's like to be on the other side of a driving scenario. You've done all sorts of things. Tell us, what is Happy Finish?

Daniel: So, Happy Finish -- or HF, as we're now more commonly referring to ourselves -- we're really in the space of creating content and experiences for grand clients, right from the very beginning. And this is second hand information; I joined Happy Finish about six, six and a half years ago. Now the business is fifteen years old. We have been employing post-production and CGI techniques to create versions of the real world -- or different flavors of the real world -- over the last six to seven years. I think we built a reputation in the immersive technology space. We started working really early on with the DK-1 from Kickstarter, testing, playing around with Unity, seeing what we could do, up to now where we're easily 200+ commercially funded, brand funded XR experiences. And we work across the whole gamut of immersive tech, from 360 video -- which is less the flavor of the month now -- through to real time based experiences across Microsoft Hololens 2, and in AR.

Alan: I've got a call one thing, that I saw here on your site, and I just got to ask: VR bungee jumping?

Daniel: [laughs] One of my crazy ideas. It was in the context, we were chatting -- watercooler moment -- chatting about how we could make some noise about untethered VR, particularly around the Oculus Quest. And it just dawned on me as well, what better way to really test what can be done in untethered VR, than tether it to a bungee rope and have somebody jump off a bungee platform?

Alan: So were you the first one to try this?

Daniel: I wasn't, actually. We a couple of guinea pigs who put their hand up first, but I did try it in the end. It was not without its challenges. We were really putting the tracking system on the Quest to its absolute limits. And so on a few occasions, it was bungee jumping with a blindfold, rather than in VR.

Alan: [laughs] Still pretty amazing that you even bit that off. So was it a brand activation, or was this just an internal--?

Daniel: This is an in-house piece for us. And back until very recently, an area of focus for us was LB. So we've created a number of -- I think -- pretty recognizable motion platform based VR experiences. One that I know got a lot of sharing couple of years ago, that lives at the top of The Shard -- the tallest building here in London -- where the user will fly virtually -- or slide virtually -- around the top of The Shard. We got a lot of screamers and yeah, we had a real focus on building out that as a product. We have four or five of these VR slides around the world now. However, LB has taken somewhat of a hit -- as you can imagine -- based on C-19. And I think there are numerous challenges ahead in terms of expecting audiences to go to a physical location and put on a headset, not least from a sanitary perspective.

Alan: Companies like Cleanbox are cleaning up, so to speak.

Daniel: Yeah, absolutely. [laughs] And that sanitary component of VR in public has always been a piece which needs needs careful consideration.

Alan: Yeah. I mean, but the thing is, everybody that I know that's been in LB or that has been doing this awhile, they are either using alcohol swabs or-- there's a number of different things you can do to keep people safe. And I've not heard of one infection caused by VR headset yet. So it's something that you need to address and take precautions. But I've never heard of it ever being an issue.

Daniel: No, totally. I mean, I guess the key thing is to instill that trust in the user base, to make sure that they are comfortable and confident.

Alan: This sky slide that you built, if you want to see it, it's happyfinish.com, and there's a video there. It's truly amazing. You took The Shard and built this fantasy land slide that you go around. It's just incredible. I'm looking at all these different things that you've done, and it's such a wide variety, from art museums to mindfulness to Ford wheel swap. What happened with the Ford wheel swap?

Daniel: It's one of my favorite projects, actually. And there have been a lot over the last six, seven years or so. We've been working with Ford in Europe for a handful of years now. And alongside Hill+Knowlton Strategies, one of the WPP [Group] PR and strategy agencies. We were engaged in more corporate social responsibility work for Ford here in Europe. I'm a cyclist. I'm also a driver as well. And I'm very conscious of the fact that actually when I'm in either of those shoes, my feeling towards other road users changes somewhat. Irrational, really, because it's just a trait. So is there something we can do using the empathy power of VR to put people in the shoes of another road user at that point of conflict, to understand it from their perspective? So that's what we did. We captured two different tracks of content in 360 we shot in Barcelona, and at those points of conflict, we switch the perspective for the user, to see it from the cyclist's or the driver's. And it's not always the drivers at fault. Cyclists can be pretty crazy people at times. I think what added some power there was working with behavioral scientists, to actually measure the impact of the content in the long tail. So the sort of questions of "How do you feel after the experience? Will it encourage you to change your behavior?" And then checking back in later to see if it had stuck with people, resonated with people when they had changed their behavior. I think going back to maybe 4,000-5,000 people that ultimately had the experience, 60 percent of those said they had changed their road using behavior as a result of the experience. And for me, even if we manage to save one person from ending up in hospital, then that's a real result.

Alan: The problem with these things that I see is that, yes, that's a real result, but how do you measure it? It's one of those things that's impossible to measure. And so what I've noticed with businesses -- especially after this whole pandemic -- is that businesses are really focused on what can this technology do to solve my problems today. And so I know you are working on some other things that are kind of more on the functional practical side of things, rather than marketing. What are some of the other things that you're seeing, that is driving businesses to deploy this technology?

Daniel: That's a good segue in, actually, to some of the more business use cases that now we're exploring pretty heavily. And I think we're leveraging the hard earned skillsets around understanding how to build a spatial experience and understanding how to create digital twins of products, environments, even people. One of the areas-- and I'm hopeful that this is an area where in the new normal in the future, we can bring some positive societal impacts. We are, as an example, working with one of our key clients and to develop digital versions, virtual versions of their future products, removing their reliance on physical sampling. Many, many businesses in the fashion and apparel space rely on samples being made -- normally in the other side of the world -- being shipped to head office for their retail partners, sales teams to be able to showcase those products to their network of resellers, only for those products never to actually see the light of day. In fact, sometimes in the industry, in the fashion industry, those products are actually incinerated from a precautionary perspective in terms of the sensitivity of those products. So digitizing and virtualizing that supply chain piece can bring pretty big benefits, both from an economical perspective, but also many, many big global brands now need to pay a lot of attention to the damage they may have done previously, or how they can improve their impact on the environment. And I see some real scope there, through digital twinning, to improve that.

Alan: I've seen this also. One of the biggest issues that manufacturers of apparel and shoes especially is-- my daughter actually owned a shoe company. And I mean, she was only making two styles of shoes, and it took five iterations of design. So when I say five iterations, I mean, she would design it, send it to China, they built a sample, sent it to Canada, we looked at it, sent the changes back. Back and forth five separate times to get it right. And I mean, that's costly. And that's two designs on a small scale. Imagine somebody like Nike or Adidas having tens of thousands of samples a day, going back and forth.

Daniel: I've heard it said that Nike sampling business is the third largest manufacturer of footware on the planet.

Alan: Just their sampling.

Daniel: I don't-- I'm not sure how true that is.

Alan: And I think they realize that that's not sustainable anymore. And everybody is looking for solutions, which is fantastic. I don't typically plug my own products on here, but I have to say, we may have a solution for Real-Time Communications for this.

Daniel: I think you may well.

Alan: Yeah. I really think we can solve some of these issues, anyway, by allowing people to collaborate in 3D, real time. I mean, there's VR collaboration tools and stuff, but 99.9 percent of people right now are still on on a computer or a phone or tablet. So we have to go where people are, and then move them towards where we want them to be. If you want to check out that, it's metavrse.com. And I won't talk anymore about that.

Daniel: [chuckles] Nobody is really irrelevant. We're at a point in time where the developments in real time rendering-- we just have to look slightly ahead to Unreal 5 and its ability to handle a vast variety of CGI assets, which would normally, in previous times, need a lot of optimization to work in engine. So there's that, plus 5G technologies, and also machine learning to improve the speed at which we can build digital representations of products at scale. Those virtual products, they play a part in the design iteration process, speeding up time-to-market. Also in the sales -- B2B sales -- process. But ultimately, those same assets can also form a really key part of the go-to-market marketing communications to end consumer as well. So I think we have to be looking at that interoperability of an asset from its inception as an idea, all the way through to consumer.

Alan: I almost feel like I paid you to do this, because we have that exact same-- we have a slide about this, that how if you build a -- we'll just use a shoe, because we've been talking about shoes -- you build a shoe for design, it goes back and forth, it becomes a shoe. Now, that same asset -- that same 3D model -- can be used for training your team on the features of that product. Now you take it from training, you put a different type of verbiage on it, now it becomes a marketing piece. You make that available to all your your teams online, on your website, and now becomes a sales piece. So we believe that there's a this kind of design-training-marketing-sales continuum, where brands typically would buy or design or create different marketing materials based on each of those needs. So the marketing department never talk to the training department on photos they needed, they just took more photos. And so there's this kind of unique opportunity, where you can build one asset and serve several needs of your business with that one thing. And IKEA does this really well, as they create everything in 3D and about-- actually more than 50 percent now, but about 50 to 60 percent of their catalog, none of the photos are real, they're all CGI. So they don't stage the rooms anymore to take the photos, because it's just costly. So you can completely stage a room virtually and then take a screenshot of it.

Daniel: Absolutely. These are business needs that, in many ways, this time of enforced lockdown and inability to go and set up those room sets physically on location, and hold those sales meetings, and designers to come together physically in the same space, and look at iterations is a catalyst in many ways for these type of technologies to come in, and actually paves the way for a more efficient future. So I'm hopeful. Positivity is my natural defense mechanism.

Alan: [chuckles] I love it. That's why you and I get along.

Daniel: [chuckles]

Alan: So much going on in the world right now. So much uncertainty. I mean, there's unrest. There's uncertainty. People are legitimately concerned about the future. And the first time in kind of 50 years where we really had a global concern for the future of of humanity. And I think it's good in a way that it's woken us all up to some big issues and challenges in the world. I mean, I watched a giant piece of land break off of the landmass and float into the ocean last night. Where was that, Julie? Norway. We have a long, long way to go, to solve a lot of these issues. But I'm confident that students and young people coming up with this technology that we're giving them will be able to solve a lot of these problems. What problem in the world do you want to see solved using XR technologies?

Daniel: Let me broaden that out, because XR technologies -- for me -- has always been a subset of communication technologies in general. And what I am-- and I think we all in this industry are starting to find is it's not just about HMDs. It's not just about delivering content in an augmented or virtual fashion. It's actually about how that content, whatever it might be -- particularly digital content, CGI -- is built and created. And can we get to a point of efficiency and scale in that creation, digital twinning, that means it is an appropriate choice for big brands to try and revolutionize their supply chain, for example? So for me, I think it's about looking at the expertise that we have gained, working at the cutting edge in this area, and then matrixing that across real world problems and thinking about the net positive effect that we can have through applying those skillsets. The one to me, for sure, is looking at the existential threat we face in terms of the damage we're doing to our planet. I've got two kids, I've got a nine year old and a six year old. I want to be able to look back at some point in time and say, "I and we in this industry had a part to play in finding a solution for this," which is probably the biggest problem we will face, coming down the pipe.

Alan: Yeah, I agree. And I think there's more and more evidence to show that we've kicked this thing off long enough. Actually, way back when I was listening to a Voices Of VR podcast by Kent Bye, one of the episodes I was listening to was from the CEO of CCP Games. They make Eve Online. And what he was saying is by having virtual worlds and being able to create a virtual world economy like Second Life, but in your real world. You put on a pair of glasses and your avatar is dressed the way you want. You've got maybe digital car, digital space. And being able to do-- to be able to satisfy your needs for fashion and your needs for things digitally -- because it costs very little to push electrons than it does to push fabric and shoes and metal -- maybe we can reduce the amount required by each person still giving them the striving that they need and the challenges that they need, but do so in a way that's really sustainable, in the fact that we can create more electrons at will. And that that, I think, is one step closer to a sustainable world.

Daniel: I would agree, I would agree. I mean, we're already seeing interesting things happening in virtual worlds. Everybody, I think, saw some point of news around Travis Scott's concert in Fortnite. It was really interesting, really interesting. And the level of engagement that it got, I think it proves out that there is lots and lots of scope for things to purely happen in a virtual way. And the level of personalization that can be achieved -- if we talk again to fashion -- without the need for things to actually be physically made, then it's-- in some ways could be seen as some sort of strange dystopian future where none of us leave our houses ever. But there's some benefit there. We've got to really be thinking about our purchase decisions in a more altruistic way in future. And we don't need the physical thing, let's not buy it.

Alan: I agree. We don't need 500 pairs of shoes in our cupboard, although some of the shoes coming out now are just super cool. [laughs]

Daniel: I mean, what we don't want to do is limit people's human desire to have nice things. We need to feel different to each other in some ways, distinctly different as a human to all of the other humans. We don't want to be homogenized. But that has to be tempered with the sort of innate need that we should all have to belong to a community. And what should come along with that need and want and desire. Oxytocin will trigger serotonin in the brain when you have social interaction. What we should consider alongside getting that serotonin here is am I actually having a negative impact on the future of this community in what I'm doing?

Alan: That leads us to something else that we just released a couple weeks ago called xrcollaboration.com. And basically what we realized three months ago, I guess, when this whole lockdown started, is there's there's over a hundred XR collaboration tools from Facial, to ENGAGE, to Meet In VR, to meetingroom.io, Rumi. There's hundreds of them, or at least a hundred anyway. And how do you -- as a business person or an organization or a school or anybody -- sift through that? How do you go "OK, well, I need it for 20 people in a classroom. Some have headsets and some don't. And we're on multiple devices." So rather than try to guess their way through that, we made an online tool at XRcollaboration.com that allows people to go through a directory, select the tools they want to use. And then we also created a huge guide that talks about everything from security, to device management, to you name it. It's tools like this that I think really are going to help people adopt this technology. And it's one thing to make technology that works. It's another thing to socialize it, so that people start to use it every day. And I think we're in that socialization phase, where everybody's got a phone in their pocket. They have it every minute of every day. How do we engage with them on that device, and then prepare for the future where it's a pair of glasses?

Daniel: Totally, totally. And that future is really not that far away. As we know, some of the big guys -- including Apple -- are really not far from market. But for me, I mean, it's-- the reason the mobile device has become such a partner in crime for us all is a utility. In another life, I should have been a UX guy and HCI guy, just through and through. But I truly believe in that the simple, useful interactions are the sticky ones, that's what drives adoption of innovation. Does it does it fulfill a need for me that is harder to fulfill in other ways? And that need sometimes can be purely entertainment. We all need to be entertained.

Alan: Indeed. Somebody said Netflix represents something like 25 percent of the entire world's Internet bandwidth? I don't know if that's true or not, but, man, that's crazy. Even if it's a tenth of that.

Daniel: Yeah, they've got the formula right.

Alan: How can people find you? And what is your ideal client look like?

Daniel: First port of call, I'd say our website. It's www.happyfinish.com. You can check out an array of the cross section of what we do from a content/experience creation perspective there. And always LinkedIn. I'm always open to speak to people on LinkedIn, who have interesting problems that we might be able to apply our skillset to solve. We're pretty fortunate. We've already got some pretty great long term clients that we really enjoy working with. But really now, as we're realigning and focusing our expertise on the positive impact digital twinning and virtualization of products can have, then any business, any brand that is in the business of building, creating, and then selling products to the wider world. I think we've got some solutions that we should talk about.

Alan: So, yeah. Thank you, Daniel, so much for joining us today. We're excited for the near and long term future of this technology.

Daniel: We're in a good space.

Alan: It's been a wonderful conversation and I hope we get to continue this in-person soon over a pint. Thank you so much for joining the podcast today.

Daniel: Thanks for having me, Alan. I look towards that pint at some point soon.

Alan: Thanks for listening, everybody. That's been the XR for Business Podcast with your host, Alan Smithson, and my guest, Daniel Cheetham. Don't forget, if you haven't subscribed, hit the subscribe button so you don't miss out on any future episodes. It's xrforbusiness.io.

With the recent launch of the MetaVRse 3D content creation engine, Alan chats with Sikaar Keita from Oracle’s XR LAB about what he's building and why he thinks it’s a game-changer.

Alan: Hey, everybody. My name's Alan Smithson, the host of the XR for Business Podcast, and today we have a very special guest, Sikaar Keita from Oracle. Sikaar is the customer experience specialist and XR lab leader, based in France. And today, we're going to discuss how Sikaar is using the new MetaVRse web-based 3D creation platform to invent the future of customer experiences. All that more coming up next, on the XR for Business Podcast. Sikaar, it's such a pleasure to have you.

Sikaar: Thank you very much.

Alan: Today is a very special day for me personally, because we're going to talk about something near and dear to my heart, the MetaVRse platform. We've been working on this for four years, and we finally came out of beta, and we released it to the world June 1st at AWE this year. But while it was in beta, you got in there and you started creating some crazy things. And I really want to dive in there. But let's maybe just have in your own words, what is the MetaVRse platform and why did you choose to start working on it?

Sikaar: So basically, it comes with two main things that were really key to us. The first one is that it's a WebXR tool. And we really believe that leveraging the web to deploy XR experiences is really what will drive adoption. Also, it was super easy to use, and the way we were prototyping really changed leveraging this platform, compared to what we usually do with other platforms.

Alan: So what is the normal path to prototyping, compared to this workflow?

Sikaar: Normally what we would do is that we would do sketches like most design agencies would do. And we sometimes leverage 360 degree virtual tours, because it's super easy to showcase something very quickly to stakeholders, which has a sufficient quality to make them understand what the project is about. And what we figured out is just that leveraging MetaVRse, it was extremely fast to build things that looks good. So we are usually buying our models, we don't model everything ourselves. But it was super easy to come up with something that was good enough to be seen, and to help us get budget, or get the stakeholders' approval to move forward. And that was really a game changer for us.

Alan: The amazing thing that we noticed is that you hadn't been on the platform more than a few days and you started doing things that we had never contemplated ever being done. And one of them blew our minds: you connected an IoT sensor to our web-based 3D platform. Can you tell us, what was the thought process behind that? How did you come up with that?

Sikaar: So basically what we do in the Oracle XR lab is testing, prototyping, and training every possible integration of XR technology with Oracle technologies. And so in this endeavour, we emphasize what comes with feeding XR experiences with data, but also getting data out of XR experiences into record systems. So it just felt normal to get this IoT device connected with an XR experience, so we can drive the experience from out of it, and also get the data back. So basically the idea behind it is very known in the engineering industry and it's all about digital twins. But we have our own platform for that at Oracle. And the idea was can I connect it with MetaVRse, and especially how long would it-- actually, it was done in a flash.

Alan: What made it so quick, though? One of the things that I didn't understand is how did you connect it? Is it because of the ability to code in it or...?

Sikaar: Yeah, it was partially about the ability to code. But one thing that was making it super simple is that it's natively using JavaScript. So through the API -- I don't want to make it very technical -- but it was super easy to leverage all the APIs available and to connect it. Compared to what we've been doing with more classical 3D engines, it was just a matter of few hours of development.

Alan: What would this typically take, if you were using something else? What is the time comparison? Are we talking days or...?

Sikaar: Really we're talking about days, especially in the process of building the code. You need to compile it, then to make a build, then to release it. And that can be quite time consuming. One of the key things is that with MetaVRse, you don't have this build time. Basically whatever you do, literally the next second you can try it, and eventually debug it if you need to. But that's super fast process. And moreover, I really believe this is to be reminded to everyone: it is WebXR based, meaning whatever you release, the next second it's available to everyone, basically.

Alan: Yeah. I think this is one thing that we -- having done over 100 different products for different companies around the world -- nobody likes pushing apps, because there's always the delay. If you need an app for a launch of something and you're sitting on pins and needles because Apple hasn't approved it. With this, you just literally hit "save" or "publish" and it's out there. So I think that's a real bonus. And that was one of the things that we really strive to have that. It's interesting that we just added a new feature this week, that I think you're really going to love. I'll show it to you later. But talk us through, you were using a Nordic Thingy IoT sensor. Can you maybe just walk us through how do you connected that?

Sikaar: So basically, this is a web Bluetooth device that is really used for prototyping and I would invite any geek around to try what it can do. And basically, it's allowing you to capture a lot of metrics, should it be temperature, or humidity, and many other metrics that you can feed in to any system. But feeding it into an XR experience, and if you've got a digital twin of a real object -- anything in real life -- and you're just figuring out to project it into XR, this comes to be really impressive. And in the demo we built, basically we are driving the object. So whenever you turn the device, it's turning the view. Actually, we are now working on the feedback, which would be hugest things in virtual reality and tracking in the sensor in real life.

Alan: That's incredible. So you've got to tell everybody, what was the object that you were manipulating?

Sikaar: So in the example-- so what we do at Oracle is we had the red pump example, which is the IoT object. But, you know, I'm a geek and I love Star Wars. So I was using a Star Wars TIE Fighter.

Alan: That is super cool. I mean, why not use a TIE Fighter when experimenting? So the IoT experience that you created, I mean, that-- what's a practical application of that? How do we tie that to a practical use case?

Sikaar: We linked it actually to IoT fleet monitoring platform. So it's really like you got a fleet of objects that would embark search sensor, and really demonstrating how you would be monitoring all those objects that could be spread actually across the world in your factories. If you're into manufacturing in your car fleet, if you went to renting or leasing. And can be almost anywhere. And it was really important for us to also prove at scale that this is working. Really hard to have cars running all around. But I think digital twins spread across was kind of super easy.

Alan: It's funny, because you posted that and I was like, "Whoa, this is great. Look at this! Somebody just connected an IoT sensor to our system, and now it's working in 3D!" It was crazy. And we'll put the video in the show notes, it'll be on xrforbusiness.io. Of course, we're going to do a blog post about this, because it's just super cool and you've got to see the visuals of the TIE Fighter and the Nordic Thingy. But the other thing that you did is you took our Nike shoe configurator experience, and you built an analytics and retail optimization program out of it.

Sikaar: This is really tied to my original work at Oracle as a CX -- customer experience -- specialist, where my job is really about optimizing the experiences of customers, mostly on websites, applications. And for that, I've got two main products that we use. One is called Maximizer, and it's all about A-B testing. Basically, you can test different configurations of your website. Because the product is ready to be used within XR, it was kind of a challenge. Can I plug that to an XR experience, completely change the setup of an experience, and try to measure which version was leading to the best conversion or the best engagement. Actually, in just again, a matter of less than an hour. It was their set up and everyone entering the experience will fall in one or the other version. But on the backend, we are measuring what's working better. Which will lead to probably new ways of thinking of retails, because you can really have exactly what would fit you best as a display, as a layout, as products display for you. We can go down to recommendations and so on. So really interesting how we can optimize the experiences. For the Nike shoe configurator, what we did was we wanted to track in real time what people are choosing as options. And we found it really interesting, because today in most of the 3D engines, you have embarked really advanced analytics tools, but which are only talking about the experience itself, how many polygons do you have, eventually which scene is loaded. But within a scene it's rather hard to know in which room people are, or that kind of stuff, of specific metrics. With the tool we use -- Infinity from Oracle -- we can get all the granularity of information and then turn those virtual experience into something that is feeding your real marketing systems and CRM systems, to transform your beautiful XR experience into something that is creating value, something that means business.

Alan: It's interesting that in a time where more digital retail has occurred in the last three months, than all of human history. It's insane. Nike, the CEO was quoted as saying that they had a plan to get to 30 percent digital penetration by 2023, and they've already hit 30 percent digital transformation since COVID hit. So the world is moving digital and these tools are going to enable retailers to make better marketing decisions, better e-commerce decisions. And how else do you see this platform being used across enterprise?

Sikaar: I see it as main two ways, and this is even something that is emerging today. But with XR either you present things differently, or you do things completely differently. This is new ways to engage your customers, your prospects. And I see that really as expanding beyond just AR filters. We see other platforms really focusing on that. What I really believe is how do you engage with your customers, or do you value their times and attention? And we see more and more, as well, gamification is being mentioned everywhere. But we know that those 3D engines -- and especially MetaVRse -- can be reused for that specific purpose. So bringing more engagement, getting consumer to play more with the brand, and getting a real customer experience actually means difference. This is really where I see the future going. That being said, to get there, it's not just about building good looking experiences. As I said before, it will be all about integrating with existing system. Because if your experience leaves completely disconnected from your e-commerce system, from your logistic systems, it does not make sense. It's just like a nice painting on the wall in a museum: looking nice, but not bringing you further in your customer experience.

Alan: It's really interesting you say that, because the way we design this platform, it can be built either in the cloud -- and we can just serve it from our servers -- or it can be built on premises for whoever organization. And I think the interesting thing about that is that it can be fully customized to fit right into whatever company, your CMS system or your retail system. It's an added feature to make things look pretty and give these engaging experiences. But really, you nailed it when you said it has to -- it has to, *has to* -- connect with your existing systems, because otherwise you just have something pretty. It has to convert. What do you anticipate the conversion rates are going to be like with this type of system?

Sikaar: So we already know from TWC studies[...]they made some interesting studies that you get up to three times better retention -- so people staying longer in the experiences -- and you increase your conventional rates by up to 50 percent, just because people can try -- so they can either try on our try out -- but again, really try and get a real sense of what the product would be like, or what the experience would be like if they buy it. But what I also believe strongly is that it comes at a more very human and emotional way. You're really experimenting[sic] it by yourself. Pretty much like when you get people to try VR etc. for the first time, everyone is wowed with it. And we know that what is driving human decisions are emotions. And we will see more and more of those things coming up in the future.

Alan: Sikaar, I want to say, first of all, thank you for putting in the time and effort to innovate and develop and really push the limits of this technology and our technology. I'm the CEO of MetaVRse, so I am very biased here. But I want to say thank you for really showing us the true potential of our own platform. We had no idea that anybody would do what you did with it, and it really opened our eyes. What do you see as the ultimate potential of XR? What is the one thing that you want to see XR used for to change the world?

Sikaar: To be honest, better stuff for understanding of one's health. I really believe that you learn a lot about yourself in those VR experiences. And I really believe there is a huge potential to develop empathy. There is experiences on the Oculus Go -- I love that so much; I'm so, so sad that it will be discontinued -- but there is this VR experience called Notes On Blindness. And another one called Traveling While Black, that is really helping to build empathy.

Alan: That should be mandatory watching for every student in America.

Sikaar: I will be speaking about business most of the time, but this is really what I strongly believe in: how VR can -- and AR, but mostly VR -- can really help build more empathy and better know oneself.

Alan: I love it. Sikaar, thank you so much. How can people find you? it's Oracle.com, first of all. And people can find you on LinkedIn.

Sikaar: Yeah, I invite you all. You will find me Sikaar on LinkedIn.

Alan: Perfect. So thank you so much, Sikaar, for spending the time with us. Thank you for putting in this effort. And thank you for having the vision to push the limits of technology. You really, really opened our eyes. And actually, I want to just say, if anybody who's listening wants to try to MetaVRse engine, it's free to try. It's engine.metavrse.com. You can give it a try and publish, you can do anything. It's the full engine there. We literally launched this a month ago, so bear with us if there's any blemishes on it. But we think it's a very powerful platform. And I want to say thank you to Sikaar again.

Sikaar: Thank you for having me.

Alan: And thank you for listening to the XR for Business Podcast with your host, Alan Smithson. This has been another episode where we learn about the transformative power of XR technologies. You can learn more about this at xrforbusiness.io. And please subscribe to the channel so that you get all the updates. Thanks, guys.

In countries like China, the camera on a smartphone isn't just for fun selfies -- it's an everyday practical device, for everything from banking to shopping. Michael Agustin wants to fast track this sort of adoption in the west by giving retailers AR IoT technology with Curie.

Alan: Hey there, it's Alan Smithson with the XR for Business Podcast. And today we're speaking with Michael Agustin, co-founder and CEO of Curie. They're an augmented reality shopping platform that enables consumers to make better purchasing decisions quickly and visually. All that coming up on the XR for Business Podcast.

Michael, welcome to the show, my friend.

Michael: Thanks for having me on.

Alan: I'm so excited. We had the opportunity to meet recently at., well, these two things, really. It was the VRARA gathering in San Francisco, and then we went from there to Verizon's head office and we went for a meetup where we learned from Apple and a bunch of other people about what's coming up in virtual reality. So it was really great to meet you in person. I want to unpack what you've been doing, but let's talk about what is Curie? We're Curie-ous.

Michael: Yeah. So I see that you're AI-curious. So Curie is an AR shopping assistant that allows people to make decisions very quickly. This sort of ties in to the journey of any customer that is looking to make a purchase about any type of thing. So we're kind of giving people this on-screen HUD, to be able to make decisions on the fly about any different type of product, especially the ones that they would want to save money on. And it also enables retailers to sort of keep other apps from participating in show-rooming in their stores. So we're looking at big box stores by giving them the powers of augmented reality and enabling all of the same tools that shoppers would typically have online, offline. So things like online reviews and movies and people who bought this also buy these things.

Alan: That's really interesting. I think it's it's something that's coming really quickly. And, you know, you guys have listed on your website that you're computer vision company. Walk me through the consumer experience of this. So I have my phone. I'm in a store. I point my phone at a pair of shoes. It recognizes the shoes and says, here's some Amazon reviews on it -- or some reviews from that store, I guess -- and then here's some other things that you might like that aren't necessarily in the physical store, correct?

Michael: Yeah. So typically, you don't have these tools available to you offline. The reason why we say we're a computer vision company is that computer vision is like G.P.S. for your camera. So think about all the times you use G.P.S. and Waze and Yelp and Google Maps. We're doing that from a first-person perspective and providing the ability to sort of connect to things that are in front of you, via services and information. The reason why you would want to do this is that information is typically still not in front of you when you need it, but it's available online. So we were named after Marie Curie, who sort of dedicated her life to seeing what was invisible. And we would argue that right, now information is invisible, and you can't really see it when you need it on top of things.

Alan: Amazing. So are there any companies actively deploying this now, or is this still in early phases, or where were you in the food chain of startups?

Michael: So our lead investor is 500 Startups, and we are gaining customers through a program that's backed by Wal-Mart and Sam's Club and Tyson. We're talking to several OEMs and other types of retailers about incorporating our technology into their shopping apps.

Alan: Interesting, that's really cool. So how does it work from a business standpoint, if I want to sign up for this? Let's just take a company... I sell electronics online, or online and in-store, and I want to incorporate this. What is the process look like?

Michael: So typically, we target companies that have access to, or have built, an API, and then that API has access to things like reviews and content that we can just display as sort of like an on-screen HUD within your phone or smart glass. And that enables their shoppers to get the information that they typically would have gotten from an app or website, but affixed to the products that they see within a store. So we're looking to put, like, an AR into companies like Wal-Mart.

Alan: That'll be really cool. So basically, I'm in the store and I see the blender. I point my phone in the blender, it goes, "Oh, this is the XYZ blender." And it gives me the reviews around it, and basically augments that physical box in the stores, correct?

Michael: Yeah.

Alan: Then I can order it right from there, and I don't even have to take it home. They just they ship it to my house, and then I can go on with my day.

Michael: Yeah. Yeah. This concept is called New Retail, which was sort of coined by Jack Ma of Alibaba. Stores in China are very experiential and sort of future-facing. You would be able to just point your camera at a product, buy it, and have it ready for you in two hours.

Alan: I love that. It's super good. It's kind of like what Samsung is doing here by building their experience centers. You can't really... I don't even think you can buy the phone in the center. It's just for experiencing, learning. And then you can buy everything online. So that's exciting.

Michael: Mm hmm.

Alan: So what else do you want people to know about this product?

Michael: So I think that's the biggest thing to communicate, is how this kind of fits within your regular day. In China, QR codes are scanned a quarter-trillion times every year, and you can't really go about your day without using your camera. I think in the US and in the West, people kind of still see their camera as more of, like, entertainment as opposed to something that they can use practically like a credit card. Our goal is to try to bring that consumer behavior to the United States.

Alan: I find it really strange that we do so many things with our phones, everything from checking the weather to... just unlimited things with our phones. And yet we don't tie the camera to a real world very often. And I think... the idea of the QR code in North America, I don't know. I don't know what happened. We just missed out on it. And my daughter, a few years ago, kind of made it clear to me why, and how we build on this. She said, "well, why would I point my phone at a QR code to take me to a Web site that's listed underneath? Why wouldn't I just type in the website? I was like, "well...?" Because in China, they offer discounts that, you know, they really built a culture around pointing your phone at the QR code. You can pay for things and all that. How do you think we're gonna kind of get that culture over here?

Michael: Well, I think it's important to understand it's adoption of the technology. So the reason why China had a head-start is that, as difficult it is for someone in the West to understand Chinese? It's the reverse. Most of the Internet and the Web are written in Western languages. So the ability for the average Chinese citizen to look at something wasn't there. They couldn't memorize it. And two, rural China was unbanked. And so there were no credit cards. There was no Visa or MasterCard. So the only way to pay for something was their mobile phone. Their phone became their wallet. And that enabled QR to be adopted faster. And in addition to to that, companies like Alibaba and WeChat spent a lot of money educating the market and integrating themselves into every facet of society, from retail to advertising. And so so I think the way to bring that to the states is using the concept of a default effect, meaning that in any given situation, you have the advantage of using it. And so what we're looking to do is work with retailers and OEMs to bring Curie to their apps and devices. Just like what Alexa is to the microphone, we're looking to bring to the camera.

Alan: Well, you're no stranger to digital worlds, and in fact, your previous company, it was quite successful. You want to touch on that a bit?

Michael: Yeah. I founded a company called Game Salad, which was a tool that you could download to your desktop to be able to make iPhone and Android games. What we effectively did was we enabled the 99 percent who could not code to reach what is now 140-billion GMV marketplace of app stores, where 66 percent of engagement on mobile phones were games. And so that enabled a whole new type of creator economy for everyday folks and folks in emerging markets, to be able to make their own games without needing to hire a programmer.

Alan: That's amazing. You've had massive success with that. There's been millions and millions of games consumed with your Game Salad platform.

Michael: Yeah, about 10-million games have been generated from millions of creators hitting 100-million players per month. At one point I think we were 21 percent of games being uploaded to the App Store. So that's roughly equivalent to a WordPress.

Alan: That's incredible, man. And you're taking all of that experience and all that knowledge and moving towards the future of how we buy things. The future of e-commerce. I guess not even e-commerce. It's commerce in general. Retail. It's exciting.

Michael: Yeah.

Alan: What was the decision to kind of move from games to retail?

Michael: I was getting a lot of inbound for AR projects. And I think that what was missing for all of these projects was the AR. And so we attempted to create a company that just like how we made computer games accessible to the 99 percent to create. We wanted augmented reality accessible for the 99 percent of brands and retailers to be able to make use of. And the number one thing that is available everywhere are products. There's literally a hundred thousand things in any given place. And the statistics of you encountering that product is very high over and over again. And so we wanted to use that reproduction of things as almost like an information highway, creating a web across the things that we see, and being able to access the information when you needed it.

Alan: Well, right now, there's not any retailers rolling this out, right? So you're still kind of working on that. Who is your ideal customer? And if they're listening, how can they reach out to you?

Michael: Our ideal customers have been companies that are in retail, that have been wanting to sort of digitize their local stores. So we've been starting with the big ones to be able to prove it at scale. We're one of the few technologies that can do that without spending lots of money on, say, like an Amazon Go implementation, or electronic shelf labels, which could cost you hundreds of millions of dollars. This effectively reproduces the QR code, but in physical spaces and virtually. So you don't have to place stickers everywhere in order to get it to work. You just point your camera at things and start attaching information to it for shoppers to be able to consume.

Alan: So how are you training that? For example, if I look at a box. A box is easy, because you've got four sides, or whatever, six sides to it, and your phone can recognize those. But how are you doing something like a piece of clothing or something that may not be detectable as easy by computer vision?

Michael: So we are a hybrid solution. We also look at positioning. So just in case, we also tag the area in which that piece of clothing might be in. And so you'd be able to recall the information. It's sort of comparable to digital signage, but completely virtual.

Alan: And in these times of quarantines and self-solitude here, I guess that gives you guys a really good opportunity to really work on the tech and get some sales. And when people are ready to get back to shopping in the real world, I think this is gonna be a great tool.

Michael: Yeah, I think it's now needed more than ever. For these companies to be able to digitize, to know whether something is going to be on the shelves when you get there. You need to know whether or not a few companies are price gouging. You want to know whether or not you're getting the best prices. At the same time, you want to sort of practice social distancing. So maybe a store associate is not the best solution in this day and age. Maybe it's being able to access that information on your own. And so I think that these types of technologies can also help save lives.

Alan: I agree with you. What problem in the world do you want to see solved using XR technologies?

There's actually two problems that I was thinking of, where you'd want to be able to get at the table. One is invariably, the more that you create and affect perception, the more issues there are with privacy. And I think that that can be addressed by being able to keep information on the edge and local, so that no information's stored on the cloud. And if it is, it can be encrypted. And then two, is being able to use this technology to give other people a chance to access that information. About 300-million people are visually impaired, and you have other types of impairments like hearing. But you can convert information from one form to another if it's dynamic, if it's visual. And so I think that's a new way to kind of think about how to use XR; being able to synthesize data in a new type of mode that can be consumed by the people who really need it.

Alan: Well, that's a great way to think about it. And some great use cases there. Where can people find you?

Michael: Oh, we have offices in Austin and Seattle. So I'm from both areas. And the reason why I'm here in Seattle is that where there is oil, there are refineries; where there is data, there's AI. And so, yeah, I guess. Come check us out at Curie.co

Alan: That's C-U-R-I-E dot C-0.

Michael: Yeah.

Alan: Awesome. Well, thank you very much for joining. And that has been the XR for Business Podcast with your host, Alan Smithson. Today's guest, Michael Agustin from Curie. The future of retail is augmented reality.

With the next generation of Playstation set to hit shelves this holiday season, the big news in the gaming circuit is the revelation of Unreal Engine 5. But this game engine is good for more than just the next top video game experience. Unreal Engine manager Marc Petit explains the many other use cases this technology promises.

Alan: Hey, everyone, Alan Smithson here. Today, we're speaking with Marc Petit, general manager of Unreal Engine at Epic Games. If you're not familiar with Epic Games, ask your kids. They're probably playing a video game built on their development engine. For example, Fortnite is built by them. We'll be discussing the myriad ways 3D and XR can be used for organizations. And of course, what's coming up with Unreal Engine 5? All that more on the XR for Business podcast.

I am super excited and thrilled to invite Marc Petit on the show. Marc, welcome to the show.

Marc: Well, thanks for having me, Alan, and thanks for the introduction.

Alan: It's my absolute pleasure. This has been one of those episodes that I've been really excited. And I don't want to hear myself talk anymore, so please tell us, what is Unreal Engine, and how is it being is now OK?

Marc: Well, that's a pretty broad topic. So first of all, maybe we'll go back to the basics. What is a game engine? A game engine is a piece of software, it does a lot of things in real time. What is a game? I mean, a game is everything about a simulated world and a story mixed together. So what a game engine do, they can provide you with real time simulated worlds and/or stories. It's based on complex technology, like real time rendering and rules and physics. But at the end of the day, you can be and interact with the virtual world. And I think that's all XR is about, is empowering and connecting the virtual and the real. So game engine, we're kind of the-- think of it as an operating system for visual development. That's the software on which you can build a game like Fortnite, or you can build a car configurator. So where this lower layer of software comes as a bunch of tools.

Alan: One of the basic things that people misunderstand is that 3D is a little bit different than creating a PowerPoint deck. There's a lot more to it involved, you're rendering many more things all at once.

Marc: Yeah, no, absolutely. Again, think of it of a simulated world. I mean, good 3D, things react when you interact with them. And so you have-- somebody has to go and program those multiple layers of artificial intelligence to bring the right look and also the right behavior of things in the virtual world. So what the game engine does it makes this process very easy, makes it a cross-platform. It's a very, very involved and complex piece of technology. But for users, it's pretty simple to use and that's all we are about: making that process much, much simpler for everybody.

Alan: So one of the things that you guys just released is a video showing the new PlayStation 5 running the Unreal Engine, and the demo was -- and I'll quote -- there's billions of polygons running. I think we start with, what's a "polygon" and why is that important to be able to push so much data? And how does that pertain to a business? How would a business use that then?

Marc: Polygons and triangles, this is how we present 3D geometry in the virtual space. It's basically a way to represent the virtual world inside a computer. And why having billions matter is, if you look at the current generation of technology, Unreal Engine 4, you can get some amazing results, some photorealistic results for cars, and for buildings, and for factories. We use Unreal Engine 4 to teach to autonomous car about the real world. I mean, our representation is-- have enough fidelity that we can teach a robot about the real world using the game engine technology. But to be honest, that comes with a price. It's pretty complex to create the content for the game engine. And what Unreal Engine 5 is about is you will be able to use the data that you actually have available, without preparing it. We call this process data prep. It's very boring, it's very long. To have the real time, people today have to bake lighting, they have to bake things, they have to simplify things. And really what we wanted to do with Unreal Engine 5 give you a game engine that's going to give you interactive performance. It's going to give you this real time performance with the geometry, with the data that you have. Should it be a Lidar scan of your facility, of your factory? Should it be a CAD model of the objects you want to manipulate? Should it be a GIS database of the terrain that you want to use? So we built up some very sophisticated technology, so we can process data of arbitrary complexity. And again, the benefits for the user is just like no more preparation of data. Just drop what you had in the engine, it's going to work in real time. So that's a lot of engineering that we've been doing over the past two or three years, to fine-tune this technology.

Alan: So you mentioned a few things I want. I want to unwrap each of these, because I think it's vital. So people who are new to this: first of all, go to unrealengine.com and just take a look. Just stand in the home page and look. It's just stunning. Iy'd like, you're just sitting and watching these beautiful graphics. And some are video games, some are car renderings. There's all sorts of things. But you mentioned GIS data and CAD data and Lidar data. These are all-- these have nothing to do with games, typically. And yet you're able to ingest these and use them. So how are people using these types of data formats, for product design or-- what are they using them for?

Marc: So currently, what people do in the enterprise space, you have to come back to visual communication. It's a lot of what we do. But making process more efficient in the enterprise is sharing visuals and communicating. Right now 2D visuals or videos is kind of the preferred medium. But these medium heavily mutation, if you see a video of a building, you cannot interact with the building. A video is not like a walkthrough. Or if you want to train somebody on-- take the example of training. I mean, that's an industry that's been completely revolutionized by real time 3D and by VR. Because now suddenly you can interact with the objects, you can cognitively so much easier to learn when you can do it immersively with a VR headset. That's beautiful. So real time 3D is the next form of communication after video. It's a form of video that's interactive, where you can navigate within it, and you can interact with the objects. So everywhere you currently using text and video inside the enterprise, you will one day have the opportunity to use real time 3D. And that's what game engine gives you, it's real time, 3D enterprise. Look at the automotive industry right now, and what we do in the process. So, of course, we use game engine technology to validate design, and to get a project that is greenlit; say, "Look at this beautiful car." So before, up until a few years back, you were using offline rendering technology. I mean, we haven't invented CG, we've been using CG for 20 years in the industry, but it used to take an overnight render to see a 30 seconds video of a car, to show to your management team. With a game engine, you can actually let the management team interact with the car, open the door, rotate the car around, so that they can see and understand the design much better already when before the product exist. Those technology really help you make decisions on product that do not exist yet. Then again, you should look at the adoption of game engine technology in the automotive industry. The next thing we'll do is we'll be part of the design, because we do a lot of HMI, the Human-Machine Interface. The language and the visuals of video game is exactly the right technology to interact with a complex object, like a vehicle. And so we do a lot of design of user interfaces, and the whole experience of the car is now built with the game engine, so that when you touch a screen in the car, you get something that looks and behave like you would expect today, which is the pinnacle of high level graphics at Epic Games. So not only we help design the product and validate the product, we are part of the product. And then when the product gets built on the manufacturing line, you have a lot of training involved. You put new tools in place. So that's really content that you have about the car. You can use it to the manufacturing process, to train people. You can do corroborative design reviews. You can actually do meeting in VR, is routinely using our technology to validate all of your manufacturing processes. And again, this is a visual communication tool to support normal decision making. And also in automotive was interesting is we're completely transforming the way cars are sold. We use car configurator in concession online, and it's known that we have sold more accessory, or it helps to sell cars that actually are not on the parking lot of the dealer. If you take the latest car launch that we supported, like the Corvette C8, or the Tesla Model Y or the Cybertruck, or a number of other cars. Tesla and GM were able to offer virtual test drives, and start to make money with the car before they actually were rolling off the assembly line. So that communication tool is actually very important in marketing. It's also true for apartments in the real estate space. So you can see that you can apply real time 3D, and this is kind of a high visualization tool every way through the lifecycle of the product. And so the data that we were mentioning, like the CAD data, the PLM data, the BIM data, the GIS, all of those acronyms, these are the natural form of the 3D data that you currently find at the enterprise. So what we do is a game engine -- like Unreal Engine 5 -- will be able to basically considerably simplify the process of creating communication with this data.

Alan: Yeah, no, absolutely. So the interesting pattern with everybody using this technology is that the assets that you're building -- the car or the apartment -- they can be reused and repurposed. Because once you've built this asset, it's not like a static video where you build a video and if you have an error, you have to rerecord the whole video or re-edit it. You can use these assets for multiple things. You can use it as a design collaboration. And once the design is locked in, then it goes to production. You can take that same exact asset that you designed, and make it available as a training tool to train your sales staff. And then you can take that exact same training tool, and make it available for your buyers to make better buying decisions. So it's almost like this kind of magic tool that people haven't-- or the majority of people in the world have maybe experienced a little bit of, but is not really truly pervasive. What do you see as kind of the next step to making this so that every business in the world just says, "Yeah, we have to have this, it makes sense."

Marc: I mean, the next step is to reduce the cost and the complexity. And this is what we're trying to do with Unreal Engine 5. But I think you're right. I mean, what you're describing is what people call "digital twin." It's the fact that when you build something in the real world, you should build the same thing in the virtual world and make it part of the deliverables. I mean, it's true of a building, it's true of a car. And then right now, the offerings of digital twin in industry, they're a kind of offshoot of the legacy systems. And what really is different with game engine is the real time component, and the fact that real time brings interactivity. And the fact it's a game engine, you have support for all platform, you can run the same software on a tablet, in VR, in AR, you can run the same model and the same data on the same software in all of those platform. In the cloud. We do a lot of cloud streaming now directly of interactive content. So for me -- and of course, a very biased opinion from the game engine maker -- I think game engine technology is the right technology to implement digital twins. The value of this digital data for the enterprise is gigantic, and I think people are starting to learn that. I can give you an example in construction. We work with some construction companies. They have understood that you even make the digital version of the building part of what you sell to the customer. Throughout the lifecycle of the building, you can show your customer where you are in the process. You can show the model designed by the architects. And then the model made by the construction company is not the same. The architects only cared about the look of things. Now you have to have a more sophisticated model, that capture how you're going to build and the process of building the building. And then once the building is complete, you can actually-- the building is a living and breathing element. In it you do asset management, every door has an asset tag. It has a lot of systems, HVAC systems, all of systems, through IOT, are connected very easily. Your construction model becomes your reference from which you build, and it becomes truly stocked when you connect it to the data of the building, and so you can visualize all about your building in real time. That should be part of what you deliver to your customers. Not only you deliver the real building, but you deliver digital twin, which is a management tool. And you don't deliver just to tag assets or database of switches. You can make it a 3D interactive version of your building. It's not science fiction. And we do have customers that start committing to the delivery of digital twins.

Alan: The fact that everything is real time, and that you can make a change in one part of the world and have it reflect real time with the other participants. I mean, the easiest thing I've seen on this is that my daughter has Fortnite on her phone, and all her friends play on different devices. Some are playing on computers, some are playing on iPads, some are playing on phones, but they can all play together. And I would assume it's that simple for businesses, as well, to get in there with whatever device they happen to have, get in and interact with each other. One thing that's on here that kind of blew my mind was the work being done in the film and television industry using Unreal. And it's something that you don't think about. You're like, "OK, why would a real time game engine be used for Hollywood movies?"

Yeah, I think-- if you roll back the clock. We were lucky enough that we got involved with Jon Favreau on The Mandalorian very early in the process. And really when they came to us, "We want to do this TV show for Disney+. And we have half of the budget of a Star Wars movie, but we want to do eight hours of content." The way to achieve this was to basically use real time technologies and create-- what is expensive in movie making is the post-production. And you have to shoot on a green screen, and then you have to go off and get rid of the green pixels and put the virtual world, and look at it, "OK. This is not there." And you're iterating very slowly. What Jon Favreau wanted is -- because he had exposure to gaming technology in the Lion King project -- he said, "Can you give me a system where I can see in the camera what I'm going to get?" And so we created for him this immersive environment with our partner ILM. It was a 55' stage full of wraparound screens; many, many LCD screens, and a ceiling full of screens as well. So that would display the content and the environment of The Mandalorian on those screen. And then they would basically be used as a background, and they would be used also to light.

Alan: Ah, yes. So you would have the real time reflections, and oh! Super cool!

Marc: So no more greenscreen.

Alan: And then how did you track the cameras, then?

Marc: We use camera tracking techniques.

Alan: Because you would have to track the camera, in order for the parallax of the back screen to work, right?

Marc: Yeah. I mean, look, there was a little bit of development to be done, but we basically do it, and we call this "in-camera visual effects." So they were able onstage to have the actors stand there. Some of the sets were physical, some of the set set were virtual. We give them the tool to match perspective between the physical and the real. And then they were able to do a lot of shots without using these expensive post-production process that the film industry relies upon. As we speak, we have many, many projects to build those stages. We worked on HBO's Westworld as well. This is becoming mainstream, especially in post-COVID times. Think of it as bringing the location to the talent, instead of flying the talent to the location. Think about car advertising. Why would you send a crew to the beach to do this kind of commercial on the beach, if you can bring the beach and a bunch of screens and keep the car in Detroit?

Alan: Well, I mean, come on. There is always a reason to go to the beach.

Marc: Yeah. The agency creative director likes to go to Fiji for the shoot, but--

Alan: "We've got to fly everybody to Fiji for the week!" [laughs]

Marc: [chuckles] Yeah.

Alan: So let's talk numbers and cents. Talk about cost savings here. This is just one simple example: not having to fly a film crew to Fiji to film a car ad on a beach. That's got to be thousands and thousands of dollars in savings. Like, what are some of the ROI numbers that people are seeing using this technology?

Marc: It's hard to quantify. I mean, when trouble is involved is-- if you look at the math behind The Mandalorian, it was significant, because eight hours of content for a hundred million. Those numbers are not public, but I think I read that somewhere in the press. That's half the price of a two hour movie, Star Wars movie -- which is north of $200-million -- with virtually no sacrifice on the creative front. I think not having to do real prototypes in the industry, whether it's automative -- we work with Boeing, we work with Airbus -- the ROI is pretty obvious, looking through COVID. A game engine is not only real time graphics and real time physics, but it's also multi-user at the core. When you play Fortnite, it's 100 people. We did the Travis Scott concert recently, we had 12.3 million people watching the concert at the same time in the game.

Alan: No big deal. Only 12 million people at the same time watching-- and not only watching, but interacting with the environment. That was insane, by the way.

Marc: Yeah. I mean, even for us, it was mind-boggling. But that multi-user capability, which is not very common in desktop software. Today as we speak, you can fire up Unreal Engine. You could be firing up Unreal Engine in Toronto, I'm in Carey, North Carolina. We could be working together on the same model, because that multi-user editing is there for everybody. People do film scouting. They do design reviews. They do a lot of collaboration during COVID from the comfort of their home, using the built in multi-user capabilities of game engine. And that is priceless, that ability for people to work and share. Again, visual communication. Humans, we're visual animals. So when you can see buildings, when you can see parts, when you can see objects, when you can visualize processes, it's really, really good value. Back to construction. I mean, we've worked with Gilbane there. They do hundreds of simulation for a year, just to solve some very basic problem. Is the crane going to go through, or how I'm going to do this? But the fact they can get everybody on the same page, because they all look interactively at that simulation and they can look on PC, on just be connected to the cloud, or on a tablet. They can all participate into this work session, they can see "Oh, you're going to do this? I'm good with it. OK, that's fine. We're good. Let's go and do it." That technology is really, really interesting. Let's be honest. Today, it's a pretty involved process. And we tried to make it as simple as possible with the current generation of tool. But that's why we invented Unreal Engine 5, so that we make this process really simple, as of next year, so that we eliminate all of that overhead of preparing the data.

Alan: Let me just put a pencil in this, because if people haven't watched the Unreal Engine 5 demo video, I will link it in the xrforbusiness.io website, it will be there, because it is... Holy crap. First of all, you're showing off the PS5. I don't know what kind of magic they put in there, but the video you guys showed is just like-- I don't even know what to say. It's just mind-boggling. Take a minute to watch that. But I think the question I want to ask is, when did it become obvious that your game engine was useful for more than just games?

Marc: I used to run the Majenta division of Autodesk, so I was building the legacy -- not real time -- tools for visualization in the film and TV industry, and I joined Epic in 2016. At the time, for me it was like, "Oh my God, they're going to make it." That's why I joined. But I think it's really over the past 18 months that we've proved, we've showed it's strong enough. We've been running all of automotive marketing now. It's hard for me to think about a car launch today that doesn't have Unreal Engine based configurator. We have a half a million users in the architectural space, all the way to landscape designers and SketchUp users. SketchUp is a very, very simple tool to create 3D content for interior designers. We have a lot of those users. They use game engine technology. So I think it's really took off over the past two years. We worked hard to make the technology accessible. Not only you have Unreal Engine, but I would call out-- we created a tool called Twinmotion, that is like-- think about PowerPoint for 3D content, something that's very, very easy to use. We use game mechanics to automate a lot of processes, so that we make it easy. If you want to play and learn, or if you want your kids to learn how to create this kind of content, Fortnite has a mode where you can create your own island, create your own games. And it's also very, very easy. So we have the three levels if you want to tap into the power of that real time 3D. Fortnite Creative, consumer grade. Twinmotion, really easy to use. And then Unreal Engine, which is kind of the tools of the pro. We'll do so much more. I mean, the other thing that makes me very hopeful about the future, where we can make this technology widely accessible is AI. We know there are two things. One thing is the cloud, and the other thing is AI. So the cloud is because it's-- right now, if you want to consume interactive content, you kind of have to know which device you're going to use it on. So the game industry is used to manage, this is a game for Switch, this is a game for PS4. But in the enterprise, I mean, you want to create content that's consumed by any device. So the cloud is the way that you make the experience predictable for everybody, by either using cloud streaming or-- we do a very, very nice 3D cloud based platform -- like MetaVRse, not to mention it -- which complements Unreal Engine very well. The other thing is going to make this content much more accessible -- once we figure out the delivery through the cloud -- is going to be this technology called photogrammetry, and the fact that it's very easy, using pictures, to recreate 3D model of the real world. One of our division is called Quixel, they have this product called Megascans. And basically the idea is to scan enough of the world, then with machine learning you can recreate the rest. So we're not going to scan all the rocks in the world, but we're going to scan enough of the rocks in the world, so that with machine learning, we can create all the rocks in the world. That's true of cities. That's gonna be true of everything that God created, like nature, trees, and landscape, and environments, and biomes. So we're creating all of the content that will allow you to basically create those very, very sophisticated imagery virtually, very, very low cost. So once we figure out ways to create photorealistic content using AI, we can use AI to lower the cost. The other aspect of AI is we use AI to take an asset -- something that's photorealistic -- and stylize it, put an art direction on top of it. This process can be automated. Today, if you want to add a tree in Fortnite, we still build it by hand. But tomorrow, we'll be able to kind of Fortnite-ify your tree by running a machine learning algorithm on top of it, so that once we explain the creative direction that we want, we'll be able to to stylize assets automatically.

Alan: It's like that photo-- it's like Prisma.

Marc: Yeah, yeah. Exactly. Yeah. It's like filters. It's like Instagram filters. That's why I would say to your listener, you need to start considering it, because it's not science fiction anymore. Cloud distribution of interactive content, 5G edge networks are on the horizon. We're starting to get decent devices for AR and VR. We have Unreal Engine 5 around the corner, it's going to cut the costs of all the data prep. And then we're going to have databases and technology bringing the real world into the virtual space very easily. So I think the past two years have been impressive in terms of adoption, but I think the next two years are going to be even more. And COVID has accelerated that, because it made everything about our lives digital overnight. Therefore, everybody is testing and learning. And remote collaboration is a huge plus in interactive 3D technology. It's actually more efficient than Zoom, believe it or not.

Alan: We actually just released the XR Resource Collaboration Guide. It's at xrcollaboration.com. And what we realized is that with COVID and the ability to have these tools to collaborate, there was no real central resource for which tools to use when, and what devices do you need, and if I have half my team on iPads and half of them on here, how do I do it? So we put together this pretty comprehensive resource guide as a partnership with Accenture, Qualcomm and Deutsche Telekom. And there's over 100 tools that allow you to collaborate. And some of them are built on Unity, others are built on Unreal. But for the most part, you have these amazing tools out there and it's really an adoption issue, because the tools exist.

Marc: Yep.

Alan: One of the reasons why I started this podcast is how do we get this information out to the broader business community?

Marc: Yeah. I'd like to say a word about a topic that's very important for us at Epic Games is the necessity for open platform. As you say, XR technologies, and even if you call it the MetaVRse, we strongly believe this is around the corner of this ability to work, and collaborate, and entertain, and communicate in a digital virtual world in 3D. One thing we've learned about the mobile space is if we let people build walled garden and limit the connectivity between the platforms, like you cannot Facetime between an Android and Apple device. It's very important that we create that 3D digital realities as an open space, as an open world, and support open standards. Sometimes you get a device and you have to only have the application that come to the store with that device. But that's really something that is not going to help us.

Alan: And it's not going to help anybody. In the end of the day, if half of your staff have an iPhone, the other half of an Android, some people have PCS, some people have Macs. We already have a bifurcation of all of these. And right now there is a bit of a problem with 3D models not being standardized. And so what are you guys doing to help that? Because if I'm a business, the last thing I want to deal with is, "Okay, I need this model for this thing, and this model for this. And I need to build it over here for this."

Marc: I think we need to throw all of our collective voice behind the open source community. If you look at every one of them as the vertical limit, you kind of always have to choose between keeping your data hostage of the vendor – likely a CAD vendor -- or use open source formatting. I mean, we have sPickle, a rising format in the architectural space. We have USD in the world of entertainment. And in the world of simulation, there's a lot of USD being used. We have Cesium for GIS. I think that's really... we have Open XR, to do device-independent XR. So I think we, as a citizen of the environment has to be more demanding of our vendors. And I think we've tried to show the way in that. I could tell you what we do on the game side, in terms of open platform. But it's very important that we keep the data open and we let people compete on merit and not by erecting business barrier again. Mobile ecosystem and we all suffering from those walled garden. For example, in games, what we're doing is we have this system on Fortnite, it was the first game where an Xbox player could play with a PlayStation player, before it was not even conceivable. And the way you do this -- and we have not created a uniform idea -- but on our systems, you can log in with a Microsoft account, or log in with a Sony account, log in with a Nintendo account, and you can see your friends on Fortnite. You could be a PlayStation guy, I'm a Switch guy, we could play together.

Alan: That is what really opened that up, by the way. My daughter, she's got an iPhone and she was playing Fortnite with her friends on an iPhone. I mean, it's a little laggy, but she was able to communicate with her friends and have conversations. And not everybody has a console game. Not everybody's got a PC. It's so great the fact that it's opened up to the world. And I think you guys are really pushing the limits with that.

Marc: Actually, we just released Epic Online Services, all of that cross-platform goodness, so that you can create games that connect on all the seven platforms that we support. That is available for free to the entire gaming industry. We kind of want to talk in terms of being open, to make sure that we make those services available. It's a bunch of sourcing right now on Amazon, it doesn't cost us much. We're happy to, and it's such a strong statement towards an open ecosystem. So it's very important to us. And I think we've learned is to cover more and more of our life, our digital space. And so as citizen, is becoming much more important. There is much more at stake to have our digital platform as open and as transparent as possible on data collection, and on privacy, and on what they cost.

Alan: Now, on that note, Marc, what problem in the world do you want to see solved using XR technologies?

Marc: Communication. I mean, this is our technologies and interactive content. This is a right we want to communicate with each other in shared spaces, using content that is interactive. You remember how we were all amazed when we see this video with-- the video on YouTube, they were swiping on a magazine? Pinching on the magazine and wondering why the image would not zoom like the iPad.

Alan: Yep.

Marc: I think the next generation is going to be the same. They won't be able to accept content that you cannot interact with. Videos is never going to be enough for visual communication, and for exchanging ideas, or exchanging stories, or creating emotional or engaging content. I think XR is the new form of media for me.

Alan: When do you think we will see 3D on websites be as pervasive as we see videos now?

Marc: You know, when I started in 2016, I told myself this is gonna be a 10 year journey, and I have to pace myself to make real time 3D, interactive 3D a thing in the day to day life. Now, I believe it's gonna be sooner than this. At the consumer level, consumer businesses have problems with scaling, manufacturing devices. We're not there in the consumer space. But we are there with with XR in the enterprise today. And I think the cloud, we owe it to the computer manufacturer, Intel, Nvidia, AMD. They're doing amazing progress, bringing us to compute that we need. And I think we're about to have the low latency network with 5G. So let's talk again in two or three years. But I think we will have made a lot of progress. And I think COVID -- and again, the sudden digitalization of our lives -- is going to accelerate this even more, because a lot of things that were taboo two months ago, that have been routine now.

Alan: Well, is there any last things you want to say before we wrap this up? This has been a fantastic interview. Thank you so much, Marc.

Marc: You're welcome, Alan. As I said, it's a long game, where the promise of XR and real time 3D interactive content is there. It's obvious. Today there are limitations, so when people start to deploy it again, we have to go step by step, crawl, walk, run. This technology is going to be hugely important in the next two, three years. Now is the time to invest. But as usual, with modest expectations and a gradual approach. But I think it's the return on investment for everybody, I think, is going to be huge. Patience.

Alan: Patience, indeed.

Marc: Patience and pacing is critical to be successful in that.

Alan: We're going to go from the phone to the face in the next 10 years. Could be five. Could be three. Could be seven. Who knows? But in the next 10, that's a given. So what can we do now to drive value now while we train the world on how to build 3D spatial content?

Marc: I think you're absolutely right.

Alan: Marc Petit, general manager of Unreal Engine at Epic Games. Thank you, everyone, for listening. Make sure you hit the subscribe button. Every week we come up with a new episode, so make sure you don't miss any episodes. Marc, again, thank you so much for joining us. And thank you, everyone, for listening.

Marc: Thank you, Alan.

Why trek up to the Arctic circle to capture 360 footage of the aurora borealis yourself, when you can license stunning footage someone else already shot? That was Alan’s thinking when he availed Blend Media of their services, whose founder — Damian Collier — is our guest.

Alan: Hey, everyone, Alan Smithson here. Today, we're speaking with Damian Collier, co-founder and CEO of Blend Media, a centralized hub for all things VR and AR content. From stock 360 images and videos, right up to fully customized interactive experiences. All that and more, coming up next on the XR for Business podcast. Damian, welcome to the show.

Damian: Hey, Alan. How are you?

Alan: I'm so amazing. It's been a long time since we spoke. I think the first time we met was probably four or five years ago, when you started this wonderful journey. How did you get here?

Damian: Yeah. Well, I'm just thinking back to a panel that we did at VR LA, which must be at least three, if not four years ago. It must be about that time.

Alan: Got to be four years ago, yeah.

Damian: It's crazy how time flies. And obviously, having been in VR and AR, we've seen peaks and troughs, and ups and downs, and all in between.

Alan: That panel, because it was with-- the one with Saul Rodgers, right?

Damian: That's right.

Alan: I haven't had Saul on the show, I haven't reached out him. That panel at VR LA, to me, was like the pinnacle of the VR hype. That event was so amazing. And it seems like we kind of took a step back a bit in subsequent years, and now it's starting to pick up again.

Damian: Yeah, it was great. It was a big event. But you know what? This week I attended Laval virtually. And I think they had something like ten thousand attendees and dozens, if not up into hundreds of speakers. And I have to say, for me, it really felt engaging. I didn't attend in the headset. I attended it on the web app that they built. But it felt buzzing. It felt-- there were some great presentations and keynotes.

Alan: I have to say yes, where was: my wife, Julie Smithson, was one of them. [laughs]

Damian: Oh, great. I'm sorry to say I missed Julie's lecture, but I saw some great keynotes.

Alan: What was the highlight of Laval Virtual for you?

Damian: Well, I think it was reconnecting with people virtually. There were people there that I -- like you -- I haven't seen physically for some time. And seeing their avatars and waving at them across the room, and then kind of setting up separate meetings together, it was really well done.

Alan: Amazing. The last time I was on your site, there was Blend Media, and then it ended up being Blend Market, Blend Stock, Blend Studio. Walk us through kind of the progression of where you were when we met, and where you are now and what's what's changed.

Damian: Yeah, we started off as just a new footage business, because I am a serial entrepreneur, they say. I have businesses all in the content space. So that is my background, kind of rights management, IP. My last business was a viral video business that I sold in 2014. And I just had this idea when Mark Zuckerberg posted the first 360 video around about November 2015, I just had this idea for creating what I guess would be described as the Getty Images for 360 video. And that was the original idea. And I raised a bit of seed money, built the first platform in the first part of 2016 as a 360 video stock footage site, helping creators monetize their 360 video content through a platform.

Alan: I have to interject for a second, because we have used that as a client, and we found some beautiful, beautiful, really high resolution 360-- I believe it 8K video that we used in VR. And it was just stunning video. So I can attest to the quality on there.

Damian: Well, I'm glad you benefited from that. And that platform has grown and grown and grown, and continues to grow to this day. We've always with our content on Blend Stock been very selective about what we accept. We probably get four times the number of submissions than we actually upload to the platform, because it is premium. We always -- from the very beginning -- didn't want to include footage that was badly stitched, or not stabilized, or was produced by anyone other than professional 360 creators. And we've been very fortunate to grow that library to now what is the largest 360 video library in the world. We've got about 25-26,000 videos up on the platform. Plus, what we don't have up on the platform is quite a lot of VR films that are narrative or episodic or series that people are licensing, businesses are licensing for their platforms when they need more than sort of stock footage clips. But that business has grown and grown, we have over a thousand creators who contribute to that platform. And our clients are everyone from independent apps, platforms to Google, Facebook, Intel, Deloitte, Disney, large range of enterprise clients, who just find it quicker and more cost effective to license quality content from a platform, than go out and shoot it.

Alan: Well, especially now with this whole virus thing, you can't even-- even if you wanted to go and get the stock footage, it would be a lot more difficult than it was. So you guys are well positioned. And as an agency-- I'm not saying we're an agency, I'm just saying as an agency, you'd be looking for that, any of these shortcuts to get your job done. When we used the content, it was actually for a client. Funny enough, it never ended up seeing the light of day, the product died. But the idea was we needed a scene, it was an aurora borealis scene. And I'm not going to go way up to the north and some camera people up there. We looked on your site, found several options and found one that can match the client's colours. And so you're kind of starting to expand now.

Damian: Yeah. So we were and still do a lot of licensing for a whole range of applications, from virtual galleries, to VR for medical purposes, to enterprise training, travel, advertising, marketing. But one thing that kept coming up was that people were saying, "Well, you've got some amazing stuff. But do you have this, or could you produce this?" And we started falling into almost executive producing new production. So we did a big project for the McCallan whisky where they wanted-- they built this new-- they spent £21-million building this new distillery up in Scotland. And they have a big, big US fan base, customer base for their whisky. And obviously, you can't take all your US customers up to Scotland to see this incredible distillery that you built. So they came up with this concept of filming it in 360 and presenting it at Grand Central Station in New York, initially, in a dome and in headsets. So they commissioned us and we sort of pulled from our creative network around the world to film it and do everything else that's required from storyboarding all the way through. But I never wanted to build a production company. That was never my vision for Blend. My vision was always to become the single solution for all things, initially, 360, and now we've expanded into VR and AR. So we raised some more money from our investors, we're privately backed. We've raised about 5-million USD over the course of the last four½, five years to continue to invest in our proprietary technology platforms. And so we expanded Blend Stock and built a second platform called Blend Market. And Blend Market was intended to -- and has now achieved -- creating a scalable solution to anyone who wants to create VR or AR or 360 or XR or whatever acronym people are using these days, anything immersive. Anyone who wants to create a project or--

Alan: I have to say, I'm going to insert "XR" here. [laughs]

Damian: [laughs]

Alan: Being the XR for Business podcast.

Damian: [laughs] Well, that seems to be the acronym of the day. You might want to register all the Rs as your domain name.

Alan: I wrote an article -- you can Google it -- called "The ABCs of R", going through every single letter in the alphabet, because I just got sick of them.

Damian: Yeah. It seems to have evolved and I don't think in some ways that's helpful, because for people who are in our industry, you say "MR", they have no idea what you're talking about, or XR. I think VR and AR have become ubiquitous as acronyms now, so people do understand them. But for us, all things immersive, we built Blend Market so that anyone who wants to create a project or find a solution. Whether it's AR, filters for Instagram, or it's VR training, or it's a 360 video over the Angel Falls, whatever it is, 360 VR/AR. They upload a brief onto our platform, and all of the creators who are on the platform can communicate with them through our internal messaging platform. They can make an offer, they can set budgets, they can negotiate upload NDAs. All the things that you would have involved in the commissioning process for commissioning a new production happen within our platform. But it gives creators the ability to find new work and new briefs. And we're getting load new briefs going up there from businesses like Deloitte and Universal Music and Amazon, Audible. A whole range of other businesses that have uploaded briefs and are finding creators through the platform. And from the client side gives them options to find creators around the world, who are skilled and can demonstrate that skill through their portfolio and all the other ways the creators can show themselves off and negotiate. And through the platform they accept offers and an off they go, kind of like Upwork or some of these other freelancer platforms. But we're now also interestingly with COVID and everything that's going on in the world now, we're getting approached quite a lot for online VR training, conferencing, VR meetings, all those things that as the world has been forced to go virtual, people are exploring. So our goal really with Blend Market is to be the first port of call. Just like you want to find something out today, you go on Google. And it's your first port of call and you almost do it without thinking. That's our our vision, is to become the first port of call for anyone who is considering VR/AR, whether they are new to it or experienced in it, premium, providing that optionality. And our products work hand in hand, so if you're on Blend Stock and you're looking for a piece of 360 video footage and we don't have it, ideally, you move over to Blend Market and you'd find a creator in that territory who can film it for you cost effectively. Or if you're in Blend Market and you happen to upload a brief that says, "I'm looking for a 360 video in Times Square," if we have that video then you can license it cost effectively from Blend Stock. The two go hand in hand. So that's our vision and we're fiercely determined, and our engineering team are working furiously to develop features based on client feedback and just make this super easy, and defragment the VR and AR ecosystem. Because obviously you've got creators all over the world and you've got clients who want to do these things, and there's no real hub for finding the right creators or finding the right content. So that's our goal.

Alan: It sounds like an amazing goal. So now when I go to blend.media, I've Blend Market -- which we talked about -- Blend Stock -- which is the stock images and video -- and then Blend Studio. So explain maybe Blend Studio and how that works.

Damian: So Blend Studio was a product that we developed about two years ago, because we were starting to get asked quite a lot by clients, mostly enterprise clients: "Okay. So I have a 360 video. How do I make it interactive? How do I go from one 360 video to another, maybe through a portal or a hotspot? How do I add-- is there a way to add regular fixed frame 2D video within a 360 environment? Can I add a banner or a logo or something clickable? Does it work in WebVR, as well as in headsets?" And so we thought, well, if we can build a very simple drag-and-drop tool that allows people to do that super easily, then let's see if we can do that. So we built it. And that is Blend Studio. And there's other drag-and-drop 360 VR editors out there, that is their main business and they've tried to monetize that. But we really built it as a entry level tool for people to play around, create proof of concepts. And although we monetize it in a low level way, really it's there to help those creators and clients kind of play around and develop things. But it's super easy to use. As you say, you can find it on blend.media. And we have a freemium model for that, where you can sign up and play around with it, and develop your first few projects and publish them, and it publishes into WebVR. It also provides you with an embed code if you want to embed it on your website. And it also publishes straight into the social media platform. So Facebook, Twitter, YouTube, all the others that host and accommodate 360 video. And that is all suite of projects and products, and they all go hand in hand. We have some clients, big enterprise clients who use all three. And they use Blend Studio to create proof of concepts. it's the only 360 VR editor that is linked to content. So we've dynamically linked Blend Stock in via our API. So if you were in Blend Studio, you can either upload your own 360 video or you just search our library and pull it in and license that piece of content for your particular project. And we like to think it's just super easy to use and making people's life easier.

Alan: We need more things to make our life easier. That is for sure. And creator tools like this are definitely needed. I'm looking at the three parts of the business. Are there any companies that you're working with now that are kind of leveraging all three? Like the whole gamut.

Damian: Yeah. So we have a large enterprise client, who I'm under NDA with, but they're one of the large tax consultancy businesses. They use all three. They use Blend--

Alan: One of the big five.

Damian: It's one of those guys. And we have a very close relationship with them and with their tax team, with their employee services team. They use Blend Studio a lot for internal prototypes. They license content for conferences and events and some of their own big enterprise clients. And they have recently completed their second project on Blend Market, which was a project in China, to film something for an immersive employee product that they've developed. So obviously, as an entrepreneur -- and as a human being -- it's an absolute thrill when you've built something, or you've conceived an idea for a business and you've seen it grow, and you have a team working round the clock to develop product, there's nothing more gratifying than seeing your products used by significant businesses and solving problems.

Alan: I think as a creator of products, I think the best thing is just seeing somebody use it. [laughs] Like, "Wow! It actually got used! And it didn't break!"

Damian: [chuckles] Well, things always break, but you have a great engineering team. Really smart guys who have extensive experience, particularly in the video space. So they understand the dynamics of video. And now in WebVR and A-Frame, all the other open source platforms that we can all benefit from. And it's great that when things do break, they get fixed. And it's also great when we're constantly getting client feedback. We spend a lot of time with our clients. And it's great when a client says, "You know what would make my life even easier, or would make me consume this more, or would be really helpful? If you developed this feature or if your platform could do this." And then seeing our engineering team go off and -- subject to our roadmap and priorities -- go off and build it and release it. And it works and it pleases clients and they come back for more. That's a very special feeling as an entrepreneur.

Alan: It truly is. So on that note, Damian, I wanted to say, first of all, congratulations for getting out there ahead of the curve, raising the capital required to be successful, and then also driving this whole thing forward. So thank you for the hard work, as I know very much first hand how hard it is to build something of value in an ecosystem that is changing daily. I want to just get this from you, because you've seen probably way more of this content than most people have. What is one problem in the world that you want to see solved using XR technologies?

Damian: Wow. I mean, that's a big question. I put it into the context of the world that we are in now. You're interviewing me on a day when we're -- certainly in London here -- locked up in our houses and not able to go out and have meetings and spend time with friends, family, colleagues. I would love to see VR really create that sense of presence again, not to replace it, but in times like this, especially where you're unable to physically be together. And often I'm unable to physically be with you, Alan and Julie, because we're separated by an ocean. I'd love to-- and I've started to play around a little bit with this. There was the Laval Virtual conference, that would otherwise have taken place in France. They created it virtually and did a great job. But I've also started meeting friends and some other business colleagues in Altspace. I would love to get to the point where you can just put on a headset and feel like you are in the same room, and it creates that same level of engagement. And I really don't think we're that far off, having spent a lot of time kind of playing around with that. I think it's getting closer and closer.

Alan: I couldn't agree with you more. And I'll give a quick plug out to the xrcollaboration.com. You will find a free resource guide to all the XR collaboration tools and platforms out there. That was a partnership between MetaVRse, Accenture, Qualcomm and Deutsche Telekom and a couple of other organizations, the AWE, The Area and VRARA and CleanBox. So big shout out to the group that put that together, because to your point, being in these spaces together really is something magical. So we figured if we put together this guide, more and more people would be able to use it.

Damian: That sounds great. And as we develop our plan market, we are -- as I said -- expanding beyond connecting freelance creators with clients, to connecting solutions providers to clients. So whether it's our good friends that Talespin who do the VR training or Strivr, who also do the training or anyone else who provides the solutions, we want to be that hub to defragment the industry. So I will definitely go check that out.

Alan: Well, it's interesting, Kyle has been a guest -- Kyle Jackson from Talespin -- has been a guest on the podcast, but I still haven't had Derek Belch from Strivr on yet. So if you talk to him, send him a nudge.

Damian: I'll get him.

Alan: There's any last words you want to share with everybody, Damian?

Damian: No. I guess just in today's world, I hope everyone is as healthy as they can be. I hope everyone is staying sane and Blend Media, if we can help anyone in any way with them with VR/AR projects, we'd love to do so.

Alan: The website people can find you on his blend.media.

Damian: That's it.

Alan: Thanks so much, Damian.

Damian: Great to talk to you, Alan. All the best.

IKEA might be best known for its affordable furniture, cartoon building instructions, and hard-to-pronounce product names, but that’s not all its about. They’re also exploring how they can improve lives with XR technology, as Martin Enthed explains. 

Alan: Hey everyone, I'm Alan Smithson. Today we're speaking with Martin Enthed, digital manager at IKEA Communications, who's also part of the IKEA Digital Lab, looking five to seven years out into the future of how we bring retail to the masses. Martin is also part of the Khronos Group, an organization working on the open standards for spatial web and 3D world. All that and more, coming up next on the XR for Business podcast. Martin, welcome to the show.

Martin: Thank you, Alan. And thank you for having me here.

Alan: It's my absolute pleasure. This interview has been long overdue. You've had to get a ton of approvals and everything. So we're very, very lucky and honored to have you on the show. And thank you so much for joining us. Let's get into it. Maybe you can describe how you got to where you are, and the role that you're at with IKEA.

Martin: I started 13 years ago now in 2007, and I was hired to try to make use of computer graphics into a volume production, instead of just doing a few models or images a year or two, large volumes. And building those back-end tools, coding, setting up standards and everything, up to 2011. And then they hired me to do all development for that company, IKEA Communications. And I've been the IT manager and development manager for that all the way up to now, two years ago, when I became digital manager there. Then I headed up what's called IKEA Digital Lab, that you mentioned. Now I'm working mainly with that, looking into the spatially aware 3D future.

Alan: So how is IKEA using these tools now? Because I think it's a big shock, when you explained to me how the magazine that we get, some of the photos in there aren't real photographs, they're renders.

Martin: That story has been told a few times. But if I take it very short, it started really in volume 2012-ish. So it took like from 2007 to 2012. And in 2011-2012 we did about 10 to 12,000 high-res images a year, and I would say maybe 1,500 of them were 3D. In the last five, six years we have been doing about 50,000 high risk images a year. And about 35,000 of them is 3D, mainly the product images and those things you find there. And then, of course, a lot of kitchen brochures and such are 3D. You take a kitchen brochure from our stores and look at through that one, you will see a lot of 3D. If you take the IKEA catalog, then it's much, much, much less, because most of the time we also do video sessions in those. And that's so much easier to do in a real set. But it's a lot of 3D. But that's the offline rendering stuff, that's in huge production right now.

Alan: So that's kind of pervasive now. So when you're looking at the kitchen catalogue, most of those renters are all in 3D. It's funny, because Helen Papagiannis -- author of Augmented Human -- she's got this game, "Augmented Reality Or Real?" And I've gone through the magazine, I can't tell. I really can't tell what's real and what's 3D. So kudos to you guys for making it realistic. So we render something and we have the best quality of everything. What about real time rendering? I know a couple of years ago you guys experimented with VR and also the IKEA Place app, and real time rendering of spatial objects. What's kind of on the roadmap there?

Martin: The exploration stuff internally started already in 2010, when we made some things that was running in a browser, and then we sent off a small little file that's told how that scene was set up and then rendered with an offline rendering and sending it back again in 10 seconds. And it worked, it worked nicely. The problem is that we have so many customers on our webpage, so we can't really have a high-end computer waiting for a user like that. So that didn't really fly. Then in 2012, the IKEA catalog app had AR in it. So we used the catalog as a marker. I think we were one of the first big companies in the world using marker based AR. It was only five pages total, in the first one. And then it was added even more the next year, and the next year, and the next year. So all the way up to-- we did VR stuff with ARKit in 2017 -- if I don't say wrong -- then all the way up to there it was marker based. And then the Apple guys came with a brilliant ARKit, and it worked beautifully. And we could switch, and so we didn't need the app. And then we also made the standalone app, IKEA Place. I think it's half a year later or something like that. Then came the same in the Android version, with ARCore. That's how we've done AR so far, when it comes to older things. And then just recently now you might have read that we have made a new thing with the new Apple iPad, with the LiDAR stuff, adding functionality that wasn't possible before.

Alan: I haven't read about that.

Martin: Okay. [chuckles]

Alan: We just ordered one of our iPad Pros. For those of you who are new this, the iPad Pro 2020 edition comes with a LiDAR scanner, which allows you to kind of point cloud map your world and embed 3D objects in a very contextualized way.

Martin: Yep.

Alan: So how are you guys using that?

Martin: The way you are saying, actually. So I can't say much, as it is not that public. Not many people have got their hands on it yet, but as soon as you do, then try it. It's at least as promising as the Tango was back in the days. But now it's on a more powerful device and it can do more. So it looks really promising. We've been waiting for something like this for a long time.

Alan: So something like this -- just to kind of give people an understanding of how they could use it -- if the iPad scans a room and sees that you have a desk, you could put a lamp on that desk. And even if the desk had multi-levels, it would understand that, "Oh, I can put something on different levels" and that sort of thing. Are you able to then cast shadows onto stuff like that? How far are you guys taking this?

Martin: I will say it depends on how detailed we can make the dimensions. We haven't tried that yet. But it would be possible to do occlusion, that's at least theoretically possible.

Alan: Super exciting. I keep seeing these videos pop up of people doing occlusion in their living rooms and stuff like-- man! Our iPad can't come fast enough.

Martin: [chuckles]

Alan: You are a part of the Digital Lab looking out into the future. So you're also part of the Khronos Group, which is an organization working on the open standards of the 3D Web.

Martin: Yeah.

Alan: Maybe you can talk about how the vision of the future-- you've been doing this since 2007, you've seen 3D morph into this accessible tool to everybody. If you'd asked five years ago, "Will 3D be on everybody's phone?" I probably would have said "Not really. Here's a Tango phone and good luck." But every phone in the world now has the ability to show 3D, and now AR. You must be looking out into head-worn devices as well.

Martin: Yes, definitely. The thing is that we truly believe that interaction with our customers will increasingly be spatial or 3D-based. That's also, in a way, needed for us to reach the many people that we want to reach. That's the whole idea now, trying to increase the number of people we are reaching from about, I would you say a little bit under a billion, to maybe three times that in a few years. And it's hard to do that with physical spaces only. So that's one of the things that we need to do. And if you want to interact with us from your home and understand the things that you are able to buy, then it has to be spatial in some way.

Alan: So what you're saying is you want to bring the Billy bookcase to everyone?

Martin: Yeah. [laughs]

Alan: [laughs]

Martin: I think we almost have done that. But it's selling a lot. [laughs] The thing is that actually IKEA's vision is to create a better everyday life for the many people. It doesn't, in the vision, say anything about furniture. We do it by doing well-defined furniture for people with thin wallets. That's the whole idea. But in the vision, it's nothing about furniture. It's a lot about many people. So going back to 3D and computer graphics, the thing is to be able to do this, we have to handle a lot of 3D data. Of course, moving 3D data between different applications has always been hard. Back in 2007 and -12 and -14 and -15, and still being hard all along, because nobody's really cared about the problem. That's what is my point of view.

Alan: I would have to second that. I actually have a slide in one of our decks, and it says "3D is a pain in the assets."

Martin: [laughs] Yeah, yeah. You can move the polygons okay. You can get UV coordinates over. But anything that has to do with-- yeah, maybe the diffuse texture you can get over. But anything that has to do with how the surface reacts to light -- whatever you want to call that, a shader, or a material, or a BRDF, whatever your vocabulary is -- that has never been standardized. And I've been asking for it all the way since we talked the first time at SIGGRAPH in 2013. And I think two years ago something happened, and people were starting to talk about standard surface. The MDL guys from Nvidia started talking about things. Dassault came with the Enterprise PBR. And people are talking about it now and there are movements, but it's moving a little bit slow.

Alan: And what's Enterprise PBR?

Martin: Standard Surface MDL and Enterprise PBR, if I say it right now, is different ways of describing -- in a generic way -- a material definition. So the Standard Surface and Enterprise PBR -- both of them -- in my opinion, is our-- what I call core über shaders. It's one way of describing a shader for a material. MDL is a little bit more complicated than that, in Nvidia's things. But those are at least candidates for being a generic ways of describing materials. Standard Surface is part of MaterialX, and that you might have heard of.

Alan: So somebody who -- let's say, for example -- is a furniture store, local furniture store. They want to create a 3D version of their catalog. The problem that I see is that one, where do you get the 3D models made? Two, how do you make them? Three, how do you store them? Four, how do you make changes to them? Five, what are the standard formats, and are they going to work on all--

Martin: Yeh, that Standard Surface, and MDL, and Enterprise PBR, and all that is a little far over your head. You won't get anything of that.

Alan: Exactly. So how do we get to a point where I can -- as a marketing manager of a company -- start to use 3D as a daily tool for me, without having to take 10 years of education to learn what a file format is, and how to apply a mesh onto-- This is-- it's crazy, and it shouldn't be. That would be like saying to you, "Hey, Martin, I'm going to send you a picture by a JPEG, and you got to put the RGB colors back into it."

Martin: Yep. They come in a side car. That's how it feels.

Alan: [laughs] What is--? Who thought of that? This is madness.

Martin: Yeah. That's one of the things -- if we go back to Khronos -- why we are when we went into -- now a few years ago -- to be part of Khronos. And actually since a month now or so, part of Khronos board. It's because of this, because they had a format in there. It's called glTF. It's in the 3D Formats Working Group. And glTF is an open standard, fairly well-defined, physically based vendor that you can use for real time graphics on your website, or on a phone if it's an Android phone. That's a way of doing it. And the equivalent from an iOS phone is used Apple's version of USD, that Pixar has defined. So you have two formats, actually, that works on phones.

Alan: So what happened to the Open Web XR standard, then? If now we have two formats, that's not really standard.

Martin: glTF is standard, and USD is standard.

Alan: Not if you need a different format on a iPhone.

Martin: But that you have to talk about to the people who makes the phones. [laughs]

Alan: Got it. So basically, what happens -- and I want to make this clear, because this is a problem in the world, of our world -- that everybody's moving towards the standard so that everybody can say, "Okay, glTF's the standard, we're all gonna wrap around this." And then Apple comes along and says, "No, here's a new format."

Martin: But it's better with two, than like 14, that I had four years ago.

Alan: True.

Martin: So it's a movement in the right direction.

Alan: And I've seen some USDZ convertors come up, so that you can convert between glTF and USDZ and stuff. So there's workarounds. But that's truly-- that's like saying VHS and Beta, still.

Martin: Yeah, I agree.

Alan: Yeah. OK. I don't want to make sure that I'm clear this, because I'm still learning, and this stuff is evolving daily. So you're on the cutting edge of this. By joining the Khronos Group board now, is this-- did Apple walk away from this or did they just--?

Martin: I think Apple is part of the board in Khronos still. So they are part of that. If they want to collaborate in this or not, that's really up to them. But they are equivalent, glTF and USDZ. The version of USDZ that Apple is using is fairly equivalent.

Alan: And there's actually-- we did some research, there's about 140 3D model formats of that. There's -- like you said -- 10-14 that people actually use. But there's a lot of 3D formats. I don't know how many years it took for the JPEG to become the gold standard of images. But even if you look at that, you've got PNGs and JPEGs, and there's 100 different formats. But then one just became the easy one to send.

Martin: And then it took a few years or so for the JPEG to look the same in different browsers or different phones or anything, so just making the color come up correctly. So it takes time. And the nice thing about Khronos -- now I'm in there, so of course I'm a little bit partial in this -- but it's a group where anyone can join and everybody has an equal vote. That doesn't mean, though, that it moves fast. I think it moves fast now - at least the 3D formats -- but at least everybody has an equal vote and everybody has a say. And I think that's good. And it's not only this, it's vision processing, it's neural networks, it's parallel computing, it's virtual reality and 3D graphics. So it's many of the things that I'm super interested in, in the same organization.

Alan: It's funny you said that. I was just going to ask you, what of those things excites you the most?

Martin: All of it together. That's what makes the magic. So 3D models, of course, are the Lego pieces that we build everything on. But the neural network stuff and everything is needed to be able to do automatic things. That's also why I'm looking for those people right now. Hats out to the people, to the lab.

Alan: So let's put our futurist hats on. Let's look out ten years. I want to buy some furniture for my house. What does that look like? Or if I want to interact with with IKEA in general? Maybe it's not even furniture at this point. What does it look like when I, as a consumer, interact with the IKEA brand?

Martin: It's very hard to look 10 years out, but there are some glasses there, from somebody. Don't know who will make the first ones, that we then look back on and say that "That was where it all started," when it comes to glassware, with stereo and AR and VR in the same, of course. And if that's there, then there could be so many things happening. And then how people are using that will be the hard thing. Is this something you put on in the morning, and take off in the evening, and you never want to have off? Like people today can't even move without a smartphone. Ten years ago, people were actually using phones to be phones, to talking. [chuckles] Now--

Alan: My phone actually is the worst phone. It does everything but.

Martin: Yeah, so the people who are now raised with smartphones, I wonder if they think of them as phones you're talking mainly, or if it's a texting tool or if it's a blogger tool or whatever it is. It's something else.

Alan: I have two kids, 11 and 15, and they actually use their phones very, very differently. We've been-- we kind of give them carte blanche access to a phone. My 15 year old uses Snapchat to communicate with her friends, and they send little tiny pictures, maybe a picture of the corner of their head. It's really kind of disposable photographs, I would assume, because they're really not flattering, in my opinion. But that's how they communicate: they make a funny face and they communicate through photos, rather than typing. And then my other daughter, she's been using this app called House Party, where they have a bunch of people and it's kind like a collaboration thing, and they all get together and they talk. So watching just even kids that are a few years apart interact with devices. It's very interesting to see.

Martin: Now, one thing that I actually think will happen is that we will have-- if we want to do really augmented things in the future, we have to start looking at other things than just the visual. And that's also why I'm in the lab. We are looking at sound. We are looking at simulation of sound, and how sound changes when things are added or taken away, and all those things. We are looking at the compassionate side of AR. Let's say that you and me would like to build a space together. We have a workplace together. And if I would like you to see how this workplace looks from my point of view. I happen to be-- let's say I'm foot taller than you or I happen to be -- let's say -- colorblind. How would that space look from my point of view? And so you can feel and see that, making it the compassion thing.

Alan: Ooh, I like that. See through another's eyes.

Martin: Yeah. And then, of course, sharing it so you can be more than one person in the same space. AR is mainly single user today. And I think that's a shame, because you have to look over the shoulder on the same device. Why should you? Being able to share the same space, I think we made it public a few years ago, or three years ago. We did this with Hololens in 2017, I think. We made multi-user interaction with the first Hololens. And it works great. It's so much fun to be able to do it together, instead of doing it in single user and saving and sending over and all that.

Alan: Have you seen-- there's a company called Croquet -- that I was just introduced to -- that does server-less multiplayer real time collaborative web AR.

Martin: Yeah, there was a lot of solutions doing this, and the base of it is making some kind of anchor. I think Microsoft is calling them "world anchors" and I think Google has called them "VPS points" or-- everybody has a solution for it today or is a solution for it. And there are a lot of different companies who have solutions that claim that they can do this, but there is no open solution for it. So I can just register and' AR -- if you say so -- experience anywhere in the world, and you can go there and see it. That's not really there, and I don't know if it ever will be there, but I think it will come.

Alan: Oh, I think so. I've interviewed enough people on this podcast working on it. So I'm assuming that will be a thing where you can anchor it.

Martin: Yeah. You have to have a an open standard for anchoring. [chuckles] So that's where you store them. So that's the problem. It's all this-- having a proprietary version, no problem at all. But you will have 25 of them. [chuckles]

Alan: Yeah. This is the problem. It's interesting. We're building a guide right now to XR collaboration tools. How many XR collaboration tools do you think there are?

Martin: Counting low, 60.

Alan: There's 85 that we found.

Martin: I've seen 10 or 12, but it's normally a lot more than you see. So I would guess so.

Alan: So we're at 85. And the ones that are actually in operation, there's probably about 60. So you're bang on there.

Martin: And I would guess that more than half of them -- or even 80 percent of them -- are based on one or two of the game engines that are out there.

Alan: Actually, I would say all of them are based on Unity or Unreal.

Martin: That's what I've found, too.

Alan: Speaking of game engines, what other work is being done in the real time game rendering or web based rendering? Because right now you've got IKEA, the app. But when do you anticipate this will move into web? Does it have to happen? Does it matter?

Martin: We already have-- call them editors -- or configurators or whatever you want to call them -- on the web right now. You can plan your own kitchen and everything. And we have had that all the way since 2002 in different versions. So we have that kind of real time graphics, and have had that very long time. So those are already web based.

Alan: Is that a proprietary engine, or what is that running on?

Martin: I think the ones that we have right now are-- I think the kitchen planner is a bought system, from somebody I don't remember the name of now. And then some of the configurators are things that we have built ourselves internally, based on some JavaScript engine. I guess we do as most other companies do. We try to build our own when it's needed, building on top of things.

Alan: So looking in the future, we'll have some sort of glasses. Hopefully it'll be multi-user experiences, because that just really is exciting. And I love the fact of being able to see someone from someone else's eyes. I love that idea. You really have been in this thing from the beginning. And IKEA is more than just a furniture store, like you mentioned earlier. What are some of the environmental initiatives? Because I think this is really important for people to understand that IKEA's doing a lot more than just making furniture and selling it.

Martin: I'm not working in this area. So you're putting a little bit on the spot here. But fine. [chuckles] I think I know some of the-- at least the public stuff. And I think that it's in Japan now, that we are trying to rent out furniture. So not selling them directly. We are, of course, trying-- looking into how to build furniture out of recycled material, that we've done for a very long time. Well, we should be. I think we're almost all in place to be fully self-sufficient with electricity for everything we do, from windmills and solar cells. And we are also selling solar cell panels for roofs. I think it started in UK. I'm not sure if it'll expand. I think it's in Sweden and so on, expanding slowly into different countries.

Alan: That is absolutely incredible. The fact that a company so large has put a mission out there to make people's lives better. And looking at that holistically, I think, is the key to how we move forward, especially in these crazy times that we're experiencing now. I think a lot more consumer engagement is going to move. Obviously, it's already moved. None of us can leave our house right now. People still need furniture. They still need decorations. They still need to live their lives. So how do you feel this whole thing has contributed to the speed at which people are adopting online purchasing?

Martin: I think that in the present time, everybody is learning to become digital, even if you haven't wanted to before or haven't needed to before. So both in the workplace and privately, people are trying out buying groceries online. And they hey do everything now. And we are-- IKEA is present in almost all countries in the world. And that means that we have customers and co-workers and subcontractors and so on all over the world, affected by what's going on right now. I think this, in a way, is changing a lot of things. Let's see when we come out on the other side, where that has gone and made us become. Normally, all crises are pulling us all together, hopefully, and takes out the best in us, hopefully.

Alan: It seems like that -- especially in this industry -- it seems like everybody, when this happened, everybody's rallying together to say, what can we do to help? And I want to just give a shout out to everybody who's not just sitting on the couch watching Netflix, although there's a lot of that as well.

Martin: [chuckles]

Alan: But for everybody out there that's putting their effort in, and getting up every day, getting dressed. I know it's hard when you don't have to go anywhere, but putting in the effort to make this world a better place. And we will get through this as a community. And so I want to mention one more thing. Are you guys doing anything on smell? Because, man, wouldn't it be great to smell those those meatballs?

Martin: [chuckles] To be honest, yes. Yes, we do. We have the tactile things. And in that, we have actually pushed in touch and smell and taste. Maybe not taste, but at least smell. We sell actually a lot of products on smell, especially scented candles. But there are other things also that actually sells on smells. So if we could-- But that's really far out. But if we could have that in one way or another, that would be great. Touch is hard, also. Touch is very hard, too.

Alan: I've tried a bunch of haptic gloves so far. Have you had any really good experiences with haptic gloves?

Martin: Mmm, nope. [laughs] The thing is that grabbing something, maybe you could do. But that's not really what people are after when it comes to us. They want to feel the surface, how that flows over your fingers. That means that you have to stimulate their sensors that you have underneath the skin, because that's actually where they are at. And you have to read those vibrations in the right way. And you can actually record the vibrations correctly, but you can't play them back. It's like having a a fully fledged HDR camera on one end -- because you can capture what a human finger feels -- but then playing back with an old-time fax. That's how it feels right now, when it comes to how a surface feels. Not grabbing things. That you can, in a way, do. But that's not really what I'm after.

Alan: No, I totally get it. If we're looking 10 years out, if you want to get crazy, I mean, brain-computer interfaces are kind of the next phase of this. You should be able to encode what it feeling in your fingertips look like. But that's-- phew, we're getting some crazy territory then.

Martin: The human brain is actually super interesting. The human brain is almost like a 3D reconstruction engine. It builds a three-dimensional world for you, based on so little data. And builds that up while you are looking around and doing things. You do sit in front of a keyboard right now?

Alan: Yes.

Martin: Okay. If you look at the H key in the middle there, it's the H key and half the G key and half the J key. That's what you see on an arm's length distance actually in full resolution, 60 pixels per degree, retina display. That's the only thing you see. The rest, as soon as you go from away from the H key out to the J and K key, you are down to a tenth of that resolution. And even if you go further out, you actually don't see color in a good way. So everything else is something made up by your brain. Because you've seen it before and it actually fills in the data. It's just that center thing that we have to figure out of giving a new image to, or follow your eye movements, or something in high resolution. But the rest doesn't really need high resolution. If you can solve that, then we can do almost anything in AR. The problem now is that we don't know where you're looking. [chuckles] So we have to have high-res everywhere.

Alan: Yeah, I think this is gonna be resolved. We're starting to see the headsets come out. The HTC Vive Pro has eye tracking-- or the new one has eye tracking. The Pico Neo 2, I believe, has eye tracking option. I would assume, because Facebook has bought an eye tracking company. Apple bought SMI.

Martin: Eye tracking is the key to all of this.

Alan: It's coming. And I think one of the things that-- obviously you've tried the Varjo?

Martin: Yep. Varjo, yeah.

Alan: They took a really nice approach of affixed foviated rendering without eye tracking. But as they start to kind of figure out eye tracking, in addition to what they've got, I think they've got a really nice solution for high end. But again, it's VR and it's a giant headset that has to plug into a supercomputer. So I think the more practical use case is trying to figure out how the phones and tablets can bring this experience. And I wonder if we can use the-- we actually tried this as an experiment, putting a virtual mirror, an AR mirror on a wall, and then using the back-facing camera to project the mirror, and the front-facing camera to project your face onto the virtual mirror. But we tried this couple years ago and it didn't work. The phone actually shut itself off. So I think it was more processing, rather than the fact you could do it. It just didn't do it. [laughs] The phone went boom. So we're getting-- the processing power is getting better, and would you add eye tracking? It should be interesting.

Martin: And then the processing power we'll get in our hands for the same amount of money in 10 years will probably be a thousand times more than today. So if you look back, that's what we have been having as a development the last 10 years. So let's see what we can do with a thousand times more power.

Alan: What is one problem in the world that you want to see solved using XR technologies?

Martin: Yeah. The possibility to reach the many, with any kind of information. I think I have to expand that a little bit. The reason why I think 3D into the hands of the many will make it easier, because I think it's-- if an image says more than a thousand words, then a three dimensional thing or a three dimensional space says even more, for human. So I think that will explain whatever it is much easier to anyone. And that will also make the common knowledge about things and how things works bigger in the world. And knowledge almost always makes us smarter and better together. It's an information engine for any use. Does that make sense?

Alan: That makes perfect sense to me. And you know from the work we're doing and the stuff we've been doing, education is a very big part of why we do this. And your answer is wonderful. Thank you. If an image is a thousand words, a 3D space is a thousand images.

Martin: At least.

Alan: At least. Because it's exponential. So maybe it's a thousand times a thousand. Is there any last things you want to want to say? Where can people find more information about the work you guys are doing? Do you have a website that's specific to this?

Martin: I think if people are interested in looking into possible work opportunities, then our website is fairly easy to find. I have some public talks, you can search for "IKEA, VR, and Meatballs." And if you want to see some talks I did a few years ago, they're still valid. And I'm fairly easy to find on LinkedIn, so they can find me there.

Alan: I'm actually going to look that one up. "IKEA, VR and Meatballs." And one of the things that you guys made a while ago -- and I don't know who made it -- but it was only available in Toronto for a little bit. It was-- you could make pancakes in an IKEA kitchen in VR.

Martin: It's still, I think, available on Steam.

Alan: It was so awesome. You could run around and make pancakes. So if you have a Steam account and a VR headset, you can start to make IKEA pancakes.

Martin: I think it only works for the HTC Vive, but it's still OK. It's still out there.

Alan: Martin, thank you so much for taking the time. And I really appreciate it.

Martin: Thank you.

Labster CEO Michael Jensen was on XR for Learning not-too-long ago, talking about how XR can teach kids science in the classroom. Now he explains to Alan how that same technology is making professional training safer and more cost-effective.

Alan: Hey, everyone. Alan Smithson here. Today we're speaking with Michael Jensen, CEO of Labster, a venture backed, award winning company that focuses on revolutionizing the way science and safety is taught at companies, universities, colleges, and high schools all over the world. They started with creating multimillion dollar science labs in a VR headset. And now they're ready to take on the enterprise training world. All that and more, coming up next on the XR for Business podcast.

Michael, welcome to the show.

Michael: Hey, Alan, thanks so much, honored to be here.

Alan: It's my absolute pleasure to have you. I know you were on my partner and wife Julie's podcast, XR for Learning. And I learned all about how Labster is revolutionizing how we teach science, and making it more exciting, gamified, but also bringing the opportunity to create multi-million dollar science labs for the cost of a cup of coffee. So let's unpack that. Michael, how did you get into this?

Michael: Yeah, so that actually started about nine years ago, when my co-founder and I saw an opportunity to create much more engaging online learning content for students and learners around the world. Basically, most people were learning in very boring, non-engaging formats as we saw it. And at the same time, we saw these billions of dollars being invested into the gaming industry to create really engaging games. And we thought, why not find a way to combine and merge the learning world and the gaming world in a more engaging way, so that we can engage learners in the content, make them more excited about the topics, but also use these mechanisms to help them understand some of these more complex concepts in a much better way.

Alan: Walk people through what a typical Labster lab looks like, and why this is exciting.

Michael: There's two main components that we really looked at. One is engagement -- as I just talked about -- and the other one is timesaving, cost savings. And so what we looked at was, how can we best address some of the biggest challenges in the industry by presently creating virtual training -- similar to a flight simulator that was revolutionizing pilot training -- and then create, for instance, virtual laboratories to simulate dangerous experiments or dangerous scenarios -- like safety training -- and then that way help the universities, in our case as well as high schools -- but now also corporates -- dramatically reduce their cost and saving, as well as the time spent on this training.

And we did a huge research project now -- about two years ago -- a $6-million research project involving hundreds and hundreds of employees around the world in large pharma companies, to really analyze and understand, does this really help? Is there a way for us to create better, more engaging content? And if so, does that really help students or learners understand it better? And does it also help save costs? And the results were quite overwhelmingly positive, was published and peer reviewed -- among others -- in Nature magazine, where we saw more than a doubling of the learning outcomes, as well as engagement for learners, compared to -- for instance -- standard online e-learning training, or even personal one-on-one training. So even compared to a personal one-on-one trainer, we found that this virtual immersive training format can be far superior, both in costs, as well as an engagement, as well as the time spent on the training. So really huge, huge opportunities, really exciting.

Early results are two years ago, and since then we've really just been on a mission to create hundreds of different training simulations covering more than 18 different courses now, and now incorporate training we're especially focused on the OSHA safety training. So anything around safety training, where we can see that training is either very dangerous or very time consuming or very expensive. Those are typically the areas where we can see that virtual reality or augmented reality or even also just immersive desktop/laptop based learning can have a significant impact on the learning outcomes and results.

Alan: Where can people find more information? I'm assuming that's the SIPROS project that you were referring to.

Yeah, correct. So on labster.com/research, we publish and share all the different research studies we do. And there's more than 16 different peer-reviewed studies now. And the one-- here's especially the SIPROS related one, as well as one that was published in Nature Biotechnology, as well. It was very-- a really great study. And we have also studies comparing virtual reality to desktop based learning, to really, for us, to learn and understand, when does it really make sense to use XR -- or virtual reality in our case -- and when is it fine to just use laptop based training? Because there are specific scenarios within which virtual reality really adds a lot of value to the training.

Alan: A business person who is looking into-- maybe it's an H.R. manager or a training manager who's saying, "OK, well, I keep hearing about this virtual reality thing. Where does it make sense to engage virtual reality, versus augmented reality, versus just PDFs, or what they're already doing now, maybe video training?"

Michael: So one of the key areas we found as well in the industry is that one of the really hard things, for instance, for corporate learning is the engagement piece. So really engaging the employees and the learning is really important, so that you actually-- you get your employee-- the employees to train, and spend time on the training. And one of the big challenges there is often that the existing content is just really boring. We're talking PDFs of teaching safety training, or often very short in-person trainings that are very limited due to safety concerns, good safety concerns. So I think anyone, any learning department in a company who is considering creating more experiential learning or finding ways to engage their employees more, make them more excited about the learning, should really consider even any type of immersive learning.

So we actually-- we typically work with companies to start using laptop based training. So 3D environments, but on a laptop. So imagine it as a flight simulator, but for laboratory training or any other type of manufacturing type of training, safety training. And then start with that, because that's usually the easiest way to get started. That way you get comfortable and familiar with the technology, get it integrated to the learning management system and LRS systems and so forth. And then from there, our technology is cross platform. And then what we typically do is work with companies to slowly adopt a virtual reality component as well. So any simulation can be played either on the laptop or in virtual reality.

So once they feel like they have a lab ready, we typically start with pilot projects where we have maybe a few hundred employees go through the VR training as well, to see the results there. Then they can test it out in a more smaller scale format and get -- again -- comfortable with the practical aspects of using VR. And in terms of when it makes sense to VR, so what we see is that any type of scenario that is is hard to-- or typically requires an emotional response to really practice it well, such as -- for instance -- an explosion or a fire in a laboratory. You can imagine reading about a fire in a PDF and then you can imagine having it in a virtual reality environment. And if anyone has tried VR, you know how immersive it is and it can feel incredibly real, and you actually get an emotional response, which allows the employees to also train much better under those stressful conditions.

So a simple example, also, we have is around-- we have a lab safety simulation where there's an acid spill, where the employee accidentally gets acids into the eyes. So that's actually part of the training experiment. And they have to then -- with limited eyesight -- you have to run to the safety station, and rinse your eyes and practice the process of rinsing your eyes in VR. So you see people actually bending their head down, doing the actual body motions that are needed to perform such an important safety training. Which is something you can't really do in a real world scenario, you don't want to put acids in anyone's eyes. And it's hard to simulate limited eyesight as well in a stressful situation.

So we really try to use virtual reality, the auditory experience to recreate as close as possible to the real world scenarios. And what we see that-- the effects that we see on virtual reality compared to, for instance, laptop based training is that the the memory recall is significantly higher. And we have some specific studies that dives a lot more into the details of it. But it's really interesting to see that if you have a scenario that requires a spatial environment, for instance, where you need to practice in a specific lab layout, where the safety station -- for instance -- is in a specific area, then when you're trained in a virtual environment, you can much, much better recall the process, the steps of actually doing when you go into the real lab. So that's one of the big areas that we see the benefits of.

Another interesting concept -- this applies mostly to conceptual training -- but if you have a very complex concept, such as, let's say, organic chemistry or something-- biology, for instance, where we need to understand more complex concepts, virtual reality can really help the students understand concepts in a 3D spatial environment again. And that also is something that really helps them understand the concepts. And in some of the cases, we work with companies to train their employees and certain pharmacology research -- for instance -- so that they can better apply cutting edge research in their own research, and find new innovative ways to cure certain diseases. So there are lots of of great cases and application of VR. But I still recommend always companies to start first with the laptops, get familiar with that, just run some tests so you also get to the whole technical setup, the IT infrastructure set up to support that. And then slowly start adapting virtual reality across the different destinations. And that's driving huge results.

And even one thing I haven't talked much about here, but I can quickly mention as well, there's the increased motivation and appetite for learning that that it creates in employees is really important here as well. Many companies struggle with actually engaging their employees in learning. And it's so important for companies to stay innovative, that their employees continuously evolve their skills. And we see that when you use these immersive, engaging type of learning contents, it also makes the learners much more interested in learning more. So there is this reinforcement effect that helps the company increase their learning outcomes. So, again, any learning department who cares about their employees learning more in being engaged in the learning, I think this type of technology has a huge potential.

Alan: One of the things that we keep saying and keep hearing from the industry is that learning in general -- or education, and just training, learning -- is competing with AAA games, Hollywood movies, and social media. So unless we are as engaging or as exciting as those three, we're falling second fiddle when training people.

Michael: And actually, what you're touching a little bit up on here as well is the ability to measure the impact. If we compare it to different types of learning, it's typically very hard for learning department to measure if it really is better. And we've obviously -- given our 16 different research studies -- thought a lot about how can we actually measure the results. And what we find is that when you use these interactive type of learning formats, you can much, much better both administer, but also measure the actual impact that it has on the learners. So are they better able to perform the certain types of experiment or safety protocols and so forth? So even just using these types of trainings as an exam type training is also a possibility, where we can really measure in a much more realistic way if they're really benefiting from. We're really excited about it these days -- especially some of the companies, large pharma companies, down to even small lab diagnostics firms, for instance -- to see the impact it has both on the company but also on their learners, who constantly then want more. "What else can we now start training?"

Alan: I can imagine this is never ending. So actually, that was one of my questions. So I'm looking through your labster.com/simulations and it's mostly science simulations, cellular respiration, chemistry, diabetes, it goes on and on and on... I don't even know what "eutrophication" is... what is the process for creating a new module -- for example -- and how much of it can be reused versus new stuff? Like when a company says, "Hey, we want to recreate our lab, and here are the things we want to do?" What is the process for that?

Michael: Great question. We actually built a whole learning engine, so 3D immersive learning engine or platform, and that also means that we have a very high reusability of everything we do. So everything is actually modular. So you can imagine a little bit like Lego, you're building a house, but then you can actually reuse that and reuse it again in other simulations. So we're getting faster and faster and faster in building these. And since we have hundreds of simulations now today, every new simulation can be done -- if it's a simple one, reusing existing assets -- it could be done in as little as one day. And we actually have companies who often take our training simulations, and then they customize them, change the protocol a little bit to match their own internal protocols, or change the quiz questions a little bit to test the employees in different ways. And that can be done in as little as a few hours.

And what we do a lot now in corporate training for instances is look at the OSHA standards for safety training. Beyond a lot of the higher training we have also now fire safety, chemical safety, and blood-borne pathogens -- which is obviously super relevant these days -- and biological safety, waste disposal, emergency response, hazard identification, and so on and so on. So anything that really relates to the importance of safety within a work environment is areas that we're focused a lot on. And we actually right now, every single week, we launch at least one, typically two new simulations. So there's high demand and also high kind of throughput on our technology and platform these days.

So we would often also in some cases, if companies have specific demands, we will work with them to create an entire virtual campus. So we actually have large enterprises where we work with them to understand how do they want to train both their own employees, but also how do they want to train their customers in using their technologies and machines and equipment. And in those cases, we work very closely together with their product departments to essentially convert their PowerPoint presentation into these much more engaging, immersive formats, where we'll create the specific 3D assets and 3D machines of their equipment and so forth. And in some cases, we build an entire virtual campus. So it's sort of a virtual training campus for a large corporate. And that allows them to actually not only improve the training of their employees, but also open up a whole new revenue stream for them, so they can actually now sell a lot of this training as well to their customers or even some of their partners around the world, which is another very exciting opportunity for us that we're currently pursuing as well.

Alan: You mentioned Lego, and I understand that you guys are Danish.

Michael: Yes.

Alan: Is that like a national thing, where whenever you podcast and stuff, you have to mention Lego or another Danish company?

Michael: Yeah. So it's--

Alan: [laughs]

Michael: I feel like it! So actually, we collaborate quite a lot with Lego, so I think in our case, it's also a little bit extra close to us. But yeah, I think anyone who grew up on Lego. I'm a huge Lego fan myself. And I think to some degree it's hard to show, but I think our builder component -- which is essentially a Lego for virtual environments, training environments -- is in some way inspired by my young days of playing around with Legos, building pretty much anything my imagination could foster. And that actually reminds me of another important aspect of this: when Lego, for instance, is really known for their ability to spark curiosity and creativity in people, especially young adults, of course, but also in employees, it's very important to have-- to promote curiosity. You can imagine in the pharma industry, where you have to constantly come up with new innovations to cure certain types of diseases, curiosity is super important.

And so what we do a lot in our science experiences, the virtual reality experiences, is focusing on not just on how we train specific skills, but also how they can apply those to solve real world challenges and problems. So we're known for this in the higher ed space, especially, where we coach and mentor students on how can they apply their biology/chemistry/physics skills to solve global warming or other types of really critical problems out in the world. And you can imagine how we can do the same in corporate environments, where we work with the corporates to understand, "Well, what type of creativity or curiosity or new types of innovations are very important for them? And then how can we create training simulations that promote that curiosity by helping them see how they can connect the dots in new, innovative ways?" So when you're learning about eutrophication -- or any types of important technique or skills that they might have to learn -- how can they use electron microscopes, for instance, to come up with new ways of measuring or solving important problems?

Alan: Interestingly enough, you were part of the Educators In VR conference and people from -- I think there's 150 speakers from around the world -- met in a virtual world, in virtual reality, in Altspace, and ENGAGE, and the Rumii platform. With this whole coronavirus outbreak and conferences being canceled, this represents a huge opportunity for virtual collaboration platforms like this. And what are you seeing as the next steps with this? Are you able to -- in your simulations -- also have multi-users and have multi-person collaboration around the world?

Michael: It's a really exciting opportunity for us, of course. It's a horrible situation with the coronavirus. But it does emphasize the upsides of using virtual immersive learning, or online learning in general. And the Educators In VR was an amazing event, showcasing how you can actually host now massive online events with thousands of people in the same environments, all remote, and have a really effective collaborative communication around it as well. So we are actually focused a lot right now, especially helping high schools and entire countries right now move their learning online.

So, for instance, Japan recently just closed down all their schools, I believe it was two million or so students that were suddenly prevented from going to school. And they now have to suddenly learn from home, where they have no science facilities, obviously. So we are helping build entire virtual high school science campuses. And we do the same as well in higher ed, where we build an entire online biology degree that works in virtual reality as well. And yes, similarly now for companies, we see the effects, but it's actually already happening, because in the COVID world, we see very high cost of employees traveling from destination to destination to do the specific training, hosting the employees in these different locations, when in fact you could just ship a couple of VR headsets, or even have the online laptop based training and massively reduce costs.

So there's already incentives for companies to do this. They have-- they are already and our partners do it already to an increasing extent. And I think these types of pandemics, like the Coronavirus, will certainly promote or accelerate this adoption. And I think it will open up training departments' minds much more to the opportunities in this type of training as well as the impact. And I'm very excited about it in the way that it will help these people, who have often been maybe a little bit resistant, like, "Yeah, VR..." or "Ah, yeah, well, do we really need this type of engaging, immersive learning?" They will now have to try it out because of the conditions right now. And they will see -- I believe, at least in all cases of our customers -- they will see that the benefits are far superior than any other existing training that they have today. And at the same time, they actually save costs in this, in a big way.

There's really -- as I say -- literally no reason not to try this out and adopt this. And all the research, all the existing applications and companies using it today should be a proof point for more and more companies to try it out. So I'm excited for it, and I'm excited especially for the learners, at the end of the day. At the end of the day, we are a company that's driven by the engagement and the passion that we can inspire in all the learners around the world, to come up with important solutions to global warming and other important challenges. And we see the effects here on the learners as soon as the companies are pushed a little bit in a healthy way to promote this new engaging type of learning. We are scrambling at the moment just to keep up with all the incoming requests and demands because of this, and doing our best to to serve as many students and learners around the world.

Alan: You're doing some amazing work. How can people get in touch with you and your team, if they're interested in bringing this type of virtual learning into their classroom or into their company?

Michael: So definitely go to labster.com. And then also go to labster.com/corporate, there's a lot more information about our corporate initiatives, all the research and work that we're doing there. And there's as well a little contact form, and then fill out the form and then we'll get on a call with you and talk with you about how we can help you and support you in this transition. We have an incredible team that's guiding our partners through every single step of the process. So it's very easy to get started and it can literally be done in as little as one to two days, if people are up for it.

Alan: You've probably already answered this, but what problem in the world do you want to see solved using XR technologies?

Michael: I have a personal dream, that really is what drives every inch of my motivation here. My hope is that in 10 to 15 years, we will see the Nobel Prize winner up on stage, who's cured either all kinds of cancers, global warming, or other really important critical challenges out there in the world. And they will start by saying that it all really happened when they tried out Labster and they got inspired, they got curious about learning more and learning new skills and acquire those skills that really ultimately help them solve these important global challenges. So that's my big dream. And I believe we're definitely on the path to get there.

Alan: I don't even know what to say about that. That's amazing. So thank you, Michael, for taking the time out of your busy schedule to join us in the XR for Business podcast. And thank you, everyone, for listening. Make sure you subscribe to the podcast so that you don't miss any future episodes. This has been the XR for Business podcast with your host, Alan Smithson, and our guest, Michael Jensen from Labster.

Michael: Thanks so much, Alan.

Alan: Great, man. Thank you so much.