Sign up to save your podcasts
Or
Share Learning in Stereo, with Silicon Harlem's Ben Erwin PART 1
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Learning in Stereo, with Silicon Harlem's Ben Erwin PART 1
Volumetric video and other spatial 3D technologies allows us to add a Z axis to the way we learn. Director of Silicon Harlem Ben Erwin is on the show to discuss teaching in the same three dimensions we all live in.
Julie: Hello, my name is Julie
Smithson and I am your XR for Learning podcast host. Ben Erwin is an
expert of the XR ecosystem, including hardware and software platforms
and events in the rapidly evolving industry. Recognizing the
revolution of how we consume information being brought about by
spatial computing and volumetric video, Ben is focused on creating
new experiences in WebXR and social VR. As director of Silicon
Harlem's annual tech enabled community conference since 2018, Ben
programs the agenda, develops the marketing, and produces the events.
The next one which will be held at the forum at Columbia University
on October 16th, 2020. Welcome, Ben. Thanks very much for joining me
today.
Ben: Thank you for having me,
Julie. It's an honor to be here. I appreciate being on your show.
Julie: Great. Well, I think you
and I have had a couple of different conversations that we want to
touch base with, and starting with spatial memory. And we're in a
time where many of us are developing and working and moving to online
and virtual worlds. And I've always said -- and I think people are
starting to realize -- that we live in a three dimensional world. Why
don't we learn inside of it? And this is where spatial computing
comes into play. But even deeper from that is how we respond as a
human and how we intake this information and what we do with it. So
maybe you can do a little bit of intro to yourself and how you got
into knowing more about spatial memory and the applications around
it.
Ben: Thank you. I started my
business in the late 90s, as the Internet -- as Web 1.0 -- became a
thing. And I've been developing all the way through the rise of
social and the rise of mobile. And so this new evolution that we're
seeing right now -- some call it Web 3, some call it the Fourth
Industrial Revolution -- where we're adding 3D to-- as a medium. And
it's more than just adding a Z-axis to content. The fact that we're
inside of an experience, when you put on a VR headset and you are
inside of a room and you have other people there that are also
avatars like you, the impression that that makes on your mind is a
remarkable difference to any other type of experience. And though 2D
communications platforms -- the video conferencing, and all of the
ways that we're used to doing it -- as high quality as it is, even
the lower fidelity early stage media that we're at inside of social
VR -- and just in general -- the medium of 3D has added so much to
the way that we perceive it, that I'm very excited about how that is
going to be part of our learning process, because when we see things
and we are in an area, it imprints our minds differently than it
would when we're just in 2D space.
Julie: Just to give our
listeners a little bit more premise, "spatial" obviously
means the things that are around you. And that's not just objects,
that's your environment and also sound as well. So maybe you can talk
a little bit more about your memory and how you take these things in,
and you remember them more. That's the big change now from learning
things in two dimensions and now learning about them and being a part
of them in that three dimensional world.
Ben: Part of it -- as you
introduced there -- is that it's the room around you, it's the
objects in the room. And very importantly, it's sound. And part of
spatial audio is the perception of being close to something or what
side it's on. And it's more than just a left-right traditional stereo
audio effect that we have. It's a positional audio. But when we are
sitting down or standing inside of a VR experience, there are several
different components to that. There is a horizon. There's a-- it's
called a "skybox" or the wall that surrounds the wrapper --
if you will -- of that environment. And then all of the 3D objects in
the environment, the way that you relate to things when you see
things in stereoscopic 3D, it has a much higher impact. And a great
example is the Traveling While Black stereoscopic 360 video
experience that is available on Oculus Quest. It was a documentary
that came out for the 2019 Sundance Film Festival, right after the
Green Book movie had been nominated for an Oscar. And this was the
story of the real Green Book. And what really I found incredibly
powerful was how the presence of the people that were surrounding
you. You're sitting in a booth in a diner and you're listening to
Tamir Rice's mother explain what happened the night he was killed.
And there are people sitting on the side and the interviewer sitting
right next to you. And you're there. And if you were just looking at
that through traditional monocular video, then it would be a pretty
traditional experience. But when you are in stereo and-- I think that
the most important thing about what makes a difference is how
relative to your own size it is. So you're looking at a person and
they look proportional to your size, they don't look huge, they don't
look small. They look like they're right there in front of you. It
affects you emotionally and it embeds the experience in you, so that
you have a better retention of what is going on. But it's more than
just a stereoscopic 3D experience, because that's what they call
three-degrees-of-freedom, where you can pitch, yaw and, roll your
head. You can turn your head and everything sort of stays in balance,
but you can't move within it. Now, in a spatial experience where you
have six degrees of freedom, you also add up, down, left, right, and
back and forth. And that is really powerful, because you can change
your perspective of the room. You are the camera. And so that freedom
of choice, to be able to observe something -- and you and I had been
part of the Educators In VR a couple of weeks ago -- and this was in
an avatar, but you can move around the room, and you're listening and
you're watching a presentation on the board, and it's just like you
are there. Even if you're a cartoon avatar, it still feels just like
you're there.
Julie: So with all of these
experiences in this, the immersion of basically touching on all of
the senses, you're hearing things, you're seeing things, you're
feeling things. This probably is a great segue way into a term that
you and I started talking about more at the Educators In VR -- and I
presented it as well in my talk -- was more about the humanics, about
humans in the way that they interact with the technology, and how
they continue to feel and be socially responsible to balance all of
these things, as well as the analytics and the data analytics that
come with engaging with all of these technologies. Touch a little bit
more on that subject in. How does the mind interact with being
immersed so much, being in a three dimensional experience?
Ben: One of my favorite personal
analogies to how that works is when you've never been to a place, you
imagine what that place is like and you can see pictures of that
place and more often than not, those pictures are in 2D. And you may
have been on a plane before, so you know what it's like to fly and
you can travel there. So you can sort of imagine that. But when you
actually do it and you get on a plane and you travel and you spend
some hours and you get off and you get on a train or a car and you go
to a place and you go around, it's like-- a perfect example was a
buddy and I, when we were in our early 20s, had a pact to go to
Stonehenge. Always wanted to go to Stonehenge. It was a bucket list
item from early in my life. And so we went to Stonehenge and this was
a long time ago now, but I can still spatially re-imagine that entire
trip, including being there at Stonehenge and seeing the rocks and
moving around them and seeing the landscape, and several years after
that time I went back and visited again, and they had built it out
more of a tourist center. And so the spatial memory of my first trip
to Stonehenge, where it wasn't all fenced off and it wasn't turned
into a tourist trap. Then when I went back and it had a visitor
center and a larger parking lot and everything, that is now-- I can--
if I were to go back to Stonehenge in my mind, that whole additional
aspect of moving through it is now part of that memory. So it's not
just a photograph to me. It's a part of my psyche now, because I
spatially moved around Stonehenge.
Julie: The way that the human mind works now is obviously really interesting. And I guess that's what you call a classical humanics. And you and I talked about the Robot-Proof book by Joseph Aoun. And learning a little bit more about humanics and the model of includes two dimensions, the content portion of it, and then the three literacies. But also about the cognitive capacity, of the way that you interpret your presence. And maybe talking a little bit about the cognitive capacities of what we experience in these three dimensional experiences now in virtual worlds. What kind of impact they have on us and how we interpret them.
Ben: With cognitive capacities. I felt that Aoun's four cognitive capacities reminded me of the way that STEM is very left brain heavy with science, technology, engineering and mathematics, and that it's-- at some point, somebody added an A to STEM and made it STEAM. And there is art. And so I find that art, as much as science, is a good representation of the balance of the human mind. Between art and your creative side on one hand -- which is emphasized as part of our humanics as a need -- as well as-- there are several different levels to science that are represented in the T-E-M of STEM. And so music and physical performance and all of these are very much a part of art as technology, engineering, and math is part of science. And so I think that keeping that in balance is very important. And when people earn degrees, they earn a Bachelor of Science or they earn a Master of Arts, or a Master of Fine Arts. But it's-- if you say the Academy of Motion Picture Arts and Sciences, it's-- that is saying, "OK, here is a medium of art that includes arts and sciences" and that's sort of representative of that wholeness of the balance of the mind. And that's one of the things that really attracted me to the Aoun's concept of humanics as well as the humanities, where those are the liberal arts. And I think that the holistic thinking as part of education is very important, to be able to keep those parts of ourself in balance so that we can express ourself creatively. But all art underneath it, if you break it down, you'll find science and math, which allows you to make better art. So it's a full-- it's a continuum. And if we can feed that continuum as part of our education system, then we will do a better job of making whole students.
Julie: Couldn't agree more. with
the situation that is unfolding right now, with the pandemic and
school closures. I have an 11 and 15 year old at home now, and trying
to put program together for them to continue to be educated and
provide different methods of learning. One of the things I realized
yesterday, in my planning for them was to allow my kids to become
creative, which is something that the school systems have lacked in
the past and present. Yesterday, I allowed my kids to just be
creative and explore their creative minds, and it was everything from
baking a carrot loaf, to creating bracelets, to coloring, to
painting. And I think the STEM of creative design is in every single
one of us. But it's almost hidden. And as you say, with all of these
cognitive abilities-- capacities that we do have, being able to
balance the creative side with the technical is so important. Is
there anything else that I've just said that maybe you want to stem
off or engage further with regards to creative design thinking?
Because I think that's something that's missing a lot of our systems
today.
Ben: Well-- and design thinking is really the operative word there, because Aoun's humanics is-- it specifically mentions systems thinking, but it doesn't specifically mention design thinking. But if you combine the two of them together, then you have your arts and sciences right there. Take any creative software, take Adobe Photoshop or Illustrator, or take Premier After Effects. Very powerful creative tools. If you are using that tool and you want to be very precise, what do you do? You go to the Inspector palette and you change a number. You could draw something and have that be by hand. And that will have its own sort of aspects to it that you created with your hand, with the imperfection of your hand, or even if you have a very talented and accurate hand, it's still a hand-drawn thing. But if you are taking out a polygon tool and you're saying, OK, well, here's a square, here's a hexagon. And then you want to bend it a precise amount, or warp at a precise amount, or scale it a precise amount, then you can just put in that exact number and you can align it to guides, and all of that precision is embedded in art. And to me, that's just a great metaphor for blending arts and sciences in a creative way. There are plenty of artistic things that you can do with data, and data may seem like a very dry-- it's a concept that-- some people don't even think that much about data. But the creativity in how you can structure data to have more meaning and how to analyze it isn't just STEM.
Julie: So we're almost at the
near end of our podcast. And I think this deserves a part two of
bridging the humanics into the classroom or into learning
opportunities. Before we do sign off from this podcast, I do know
that you and I are truly passionate on humanics and being able to
balance this out. So I hope you'll join me again for another
conversation on how everyone must balance their humanics, their
creative side, their management of their data, their technology, and
remain human through this transition of being in a hyper-connected
world. So I hope you'll agree to be on a second podcast and we can
can hear your conversation there.
Ben: It would be my pleasure.
Julie: Thanks so much, Ben.
Thanks for joining me today.
Ben: Thank you, Julie
PART 2 coming soon!
View all episodes
By Julie Smithson
5
11 ratings
Volumetric video and other spatial 3D technologies allows us to add a Z axis to the way we learn. Director of Silicon Harlem Ben Erwin is on the show to discuss teaching in the same three dimensions we all live in.
Julie: Hello, my name is Julie
Smithson and I am your XR for Learning podcast host. Ben Erwin is an
expert of the XR ecosystem, including hardware and software platforms
and events in the rapidly evolving industry. Recognizing the
revolution of how we consume information being brought about by
spatial computing and volumetric video, Ben is focused on creating
new experiences in WebXR and social VR. As director of Silicon
Harlem's annual tech enabled community conference since 2018, Ben
programs the agenda, develops the marketing, and produces the events.
The next one which will be held at the forum at Columbia University
on October 16th, 2020. Welcome, Ben. Thanks very much for joining me
today.
Ben: Thank you for having me,
Julie. It's an honor to be here. I appreciate being on your show.
Julie: Great. Well, I think you
and I have had a couple of different conversations that we want to
touch base with, and starting with spatial memory. And we're in a
time where many of us are developing and working and moving to online
and virtual worlds. And I've always said -- and I think people are
starting to realize -- that we live in a three dimensional world. Why
don't we learn inside of it? And this is where spatial computing
comes into play. But even deeper from that is how we respond as a
human and how we intake this information and what we do with it. So
maybe you can do a little bit of intro to yourself and how you got
into knowing more about spatial memory and the applications around
it.
Ben: Thank you. I started my
business in the late 90s, as the Internet -- as Web 1.0 -- became a
thing. And I've been developing all the way through the rise of
social and the rise of mobile. And so this new evolution that we're
seeing right now -- some call it Web 3, some call it the Fourth
Industrial Revolution -- where we're adding 3D to-- as a medium. And
it's more than just adding a Z-axis to content. The fact that we're
inside of an experience, when you put on a VR headset and you are
inside of a room and you have other people there that are also
avatars like you, the impression that that makes on your mind is a
remarkable difference to any other type of experience. And though 2D
communications platforms -- the video conferencing, and all of the
ways that we're used to doing it -- as high quality as it is, even
the lower fidelity early stage media that we're at inside of social
VR -- and just in general -- the medium of 3D has added so much to
the way that we perceive it, that I'm very excited about how that is
going to be part of our learning process, because when we see things
and we are in an area, it imprints our minds differently than it
would when we're just in 2D space.
Julie: Just to give our
listeners a little bit more premise, "spatial" obviously
means the things that are around you. And that's not just objects,
that's your environment and also sound as well. So maybe you can talk
a little bit more about your memory and how you take these things in,
and you remember them more. That's the big change now from learning
things in two dimensions and now learning about them and being a part
of them in that three dimensional world.
Ben: Part of it -- as you
introduced there -- is that it's the room around you, it's the
objects in the room. And very importantly, it's sound. And part of
spatial audio is the perception of being close to something or what
side it's on. And it's more than just a left-right traditional stereo
audio effect that we have. It's a positional audio. But when we are
sitting down or standing inside of a VR experience, there are several
different components to that. There is a horizon. There's a-- it's
called a "skybox" or the wall that surrounds the wrapper --
if you will -- of that environment. And then all of the 3D objects in
the environment, the way that you relate to things when you see
things in stereoscopic 3D, it has a much higher impact. And a great
example is the Traveling While Black stereoscopic 360 video
experience that is available on Oculus Quest. It was a documentary
that came out for the 2019 Sundance Film Festival, right after the
Green Book movie had been nominated for an Oscar. And this was the
story of the real Green Book. And what really I found incredibly
powerful was how the presence of the people that were surrounding
you. You're sitting in a booth in a diner and you're listening to
Tamir Rice's mother explain what happened the night he was killed.
And there are people sitting on the side and the interviewer sitting
right next to you. And you're there. And if you were just looking at
that through traditional monocular video, then it would be a pretty
traditional experience. But when you are in stereo and-- I think that
the most important thing about what makes a difference is how
relative to your own size it is. So you're looking at a person and
they look proportional to your size, they don't look huge, they don't
look small. They look like they're right there in front of you. It
affects you emotionally and it embeds the experience in you, so that
you have a better retention of what is going on. But it's more than
just a stereoscopic 3D experience, because that's what they call
three-degrees-of-freedom, where you can pitch, yaw and, roll your
head. You can turn your head and everything sort of stays in balance,
but you can't move within it. Now, in a spatial experience where you
have six degrees of freedom, you also add up, down, left, right, and
back and forth. And that is really powerful, because you can change
your perspective of the room. You are the camera. And so that freedom
of choice, to be able to observe something -- and you and I had been
part of the Educators In VR a couple of weeks ago -- and this was in
an avatar, but you can move around the room, and you're listening and
you're watching a presentation on the board, and it's just like you
are there. Even if you're a cartoon avatar, it still feels just like
you're there.
Julie: So with all of these
experiences in this, the immersion of basically touching on all of
the senses, you're hearing things, you're seeing things, you're
feeling things. This probably is a great segue way into a term that
you and I started talking about more at the Educators In VR -- and I
presented it as well in my talk -- was more about the humanics, about
humans in the way that they interact with the technology, and how
they continue to feel and be socially responsible to balance all of
these things, as well as the analytics and the data analytics that
come with engaging with all of these technologies. Touch a little bit
more on that subject in. How does the mind interact with being
immersed so much, being in a three dimensional experience?
Ben: One of my favorite personal
analogies to how that works is when you've never been to a place, you
imagine what that place is like and you can see pictures of that
place and more often than not, those pictures are in 2D. And you may
have been on a plane before, so you know what it's like to fly and
you can travel there. So you can sort of imagine that. But when you
actually do it and you get on a plane and you travel and you spend
some hours and you get off and you get on a train or a car and you go
to a place and you go around, it's like-- a perfect example was a
buddy and I, when we were in our early 20s, had a pact to go to
Stonehenge. Always wanted to go to Stonehenge. It was a bucket list
item from early in my life. And so we went to Stonehenge and this was
a long time ago now, but I can still spatially re-imagine that entire
trip, including being there at Stonehenge and seeing the rocks and
moving around them and seeing the landscape, and several years after
that time I went back and visited again, and they had built it out
more of a tourist center. And so the spatial memory of my first trip
to Stonehenge, where it wasn't all fenced off and it wasn't turned
into a tourist trap. Then when I went back and it had a visitor
center and a larger parking lot and everything, that is now-- I can--
if I were to go back to Stonehenge in my mind, that whole additional
aspect of moving through it is now part of that memory. So it's not
just a photograph to me. It's a part of my psyche now, because I
spatially moved around Stonehenge.
Julie: The way that the human mind works now is obviously really interesting. And I guess that's what you call a classical humanics. And you and I talked about the Robot-Proof book by Joseph Aoun. And learning a little bit more about humanics and the model of includes two dimensions, the content portion of it, and then the three literacies. But also about the cognitive capacity, of the way that you interpret your presence. And maybe talking a little bit about the cognitive capacities of what we experience in these three dimensional experiences now in virtual worlds. What kind of impact they have on us and how we interpret them.
Ben: With cognitive capacities. I felt that Aoun's four cognitive capacities reminded me of the way that STEM is very left brain heavy with science, technology, engineering and mathematics, and that it's-- at some point, somebody added an A to STEM and made it STEAM. And there is art. And so I find that art, as much as science, is a good representation of the balance of the human mind. Between art and your creative side on one hand -- which is emphasized as part of our humanics as a need -- as well as-- there are several different levels to science that are represented in the T-E-M of STEM. And so music and physical performance and all of these are very much a part of art as technology, engineering, and math is part of science. And so I think that keeping that in balance is very important. And when people earn degrees, they earn a Bachelor of Science or they earn a Master of Arts, or a Master of Fine Arts. But it's-- if you say the Academy of Motion Picture Arts and Sciences, it's-- that is saying, "OK, here is a medium of art that includes arts and sciences" and that's sort of representative of that wholeness of the balance of the mind. And that's one of the things that really attracted me to the Aoun's concept of humanics as well as the humanities, where those are the liberal arts. And I think that the holistic thinking as part of education is very important, to be able to keep those parts of ourself in balance so that we can express ourself creatively. But all art underneath it, if you break it down, you'll find science and math, which allows you to make better art. So it's a full-- it's a continuum. And if we can feed that continuum as part of our education system, then we will do a better job of making whole students.
Julie: Couldn't agree more. with
the situation that is unfolding right now, with the pandemic and
school closures. I have an 11 and 15 year old at home now, and trying
to put program together for them to continue to be educated and
provide different methods of learning. One of the things I realized
yesterday, in my planning for them was to allow my kids to become
creative, which is something that the school systems have lacked in
the past and present. Yesterday, I allowed my kids to just be
creative and explore their creative minds, and it was everything from
baking a carrot loaf, to creating bracelets, to coloring, to
painting. And I think the STEM of creative design is in every single
one of us. But it's almost hidden. And as you say, with all of these
cognitive abilities-- capacities that we do have, being able to
balance the creative side with the technical is so important. Is
there anything else that I've just said that maybe you want to stem
off or engage further with regards to creative design thinking?
Because I think that's something that's missing a lot of our systems
today.
Ben: Well-- and design thinking is really the operative word there, because Aoun's humanics is-- it specifically mentions systems thinking, but it doesn't specifically mention design thinking. But if you combine the two of them together, then you have your arts and sciences right there. Take any creative software, take Adobe Photoshop or Illustrator, or take Premier After Effects. Very powerful creative tools. If you are using that tool and you want to be very precise, what do you do? You go to the Inspector palette and you change a number. You could draw something and have that be by hand. And that will have its own sort of aspects to it that you created with your hand, with the imperfection of your hand, or even if you have a very talented and accurate hand, it's still a hand-drawn thing. But if you are taking out a polygon tool and you're saying, OK, well, here's a square, here's a hexagon. And then you want to bend it a precise amount, or warp at a precise amount, or scale it a precise amount, then you can just put in that exact number and you can align it to guides, and all of that precision is embedded in art. And to me, that's just a great metaphor for blending arts and sciences in a creative way. There are plenty of artistic things that you can do with data, and data may seem like a very dry-- it's a concept that-- some people don't even think that much about data. But the creativity in how you can structure data to have more meaning and how to analyze it isn't just STEM.
Julie: So we're almost at the
near end of our podcast. And I think this deserves a part two of
bridging the humanics into the classroom or into learning
opportunities. Before we do sign off from this podcast, I do know
that you and I are truly passionate on humanics and being able to
balance this out. So I hope you'll join me again for another
conversation on how everyone must balance their humanics, their
creative side, their management of their data, their technology, and
remain human through this transition of being in a hyper-connected
world. So I hope you'll agree to be on a second podcast and we can
can hear your conversation there.
Ben: It would be my pleasure.
Julie: Thanks so much, Ben.
Thanks for joining me today.
Ben: Thank you, Julie
PART 2 coming soon!