Sign up to save your podcasts
Or
Share Three Decades of Medical VR, with Stanford’s Dr. Walter Greenleaf
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Three Decades of Medical VR, with Stanford’s Dr. Walter Greenleaf
XR has come a long way, baby – and we have one of the technology’s earliest pioneer’s on today’s episode. Dr. Walter Greenleaf has been working in the field for 33 years, since the days when VR was little more than a twinkle in research scientists’ eyes. Now, he and Alan chat about how far the technology has come, and how far it still has to go.
Alan: Welcome to the XR for
Business Podcast with your host, Alan Smithson. Today’s guest is Dr.
Walter Greenleaf, a behavioral neuroscientist and medical technology
developer working at Stanford University. With over three decades of
research and development experience in the field of digital medicine
and medical virtual reality technology, Walter is considered the
leading authority in the field of working in this industry, and he’s
been doing this for 33 years. Unbelievable experience. Dr. Greenleaf
has designed and developed numerous clinical systems over the last 33
years, including products in the fields of surgical simulation, 3D
medical visualization, telerehabilitation, clinical informatics,
clinical decision support, point of care, clinical data collection,
ergonomic evaluation technology, automatic sleep staging systems,
psycho-physiological assessment and simulation assisted
rehabilitation technologies, as well as products for behavioral
medicine. Dr. Greenleaf’s focus has always been on computer supported
clinical products, with a specific focus on virtual reality and
digital health technologies to treat post-traumatic stress disorder,
anxiety disorders, traumatic brain injury, stroke, addictions, autism
and other difficult problems, and behavioral and physical medicine.
He’s currently a distinguished visiting scholar at Stanford
University’s Media X program at Stanford University’s Virtual Human
Interaction Lab and the Director of Technology Strategy at the
University of Colorado National Mental Health Institute Center. To
learn more about the work that Dr. Greenleaf and his team are doing,
you can visit the Human Interaction Lab at Stanford at
vhil.stanford.edu and a new
organization that he’s formed called the International VR Health
Association at ivrha.org.
Welcome to the show, Dr. Walter
Greenleaf. So great to have you.
Walter: Thanks, Alan. I’m
pleased to be here with you.
Alan: That’s an honor. You are
considered one of the godfathers of this technology. You’ve been
working in it your whole life. And I want to personally say thank you
for laying the groundwork that allows people like myself — the new
people getting involved here — to really pick up where you left off,
and where you’ve driven this whole industry forward to, and let us
really build upon your lifetimes of knowledge. So thank you very much
for paving the way for us.
Walter: Well, thank you, Alan.
And really, without everybody else’s and my colleagues work and your
work and other people who are helping the trends position from
something that for a long time was a research lab curiosity and
something that really hadn’t escaped the confines of academia. Now we
have it out there in the world. And I’m particularly excited about
all the progress has been made in applying VR and AR technology to
difficult problems in healthcare. For me, it’s a very exciting time.
Alan: I’ve been keeping track of
all of the different things that come up in my news feed and I have a
health and medical folder. And it’s interesting, because last year I
actually had to break it apart into a mental health folder, in
addition to the traditional health and medical. So there is an
enormous amount of, not just research, but real practical
applications being created for this. You know, one of them, one that
stands out the most to me is being able to use virtual reality to
treat lazy eye or strabismus. I thought that was just amazing. Within
a few sessions, people are seeing complete reduction and elimination
of their lazy eye using virtual reality. And that’s just one of a
thousand use cases in this technology. So maybe walk us through the
work you’re doing at the Virtual Human Interaction Lab, and what are
the some of the great use cases that you’re seeing now?
Walter: OK, well, sure. Jeremy
Bailenson is the director of the Stanford Virtual Human Interaction
Lab, and I help out there as the medical advisor and expert. The
Stanford Virtual Human Interaction Lab has a focus on exploring and
studying how VR technology can promote pro-social behavior, such as
helping us better understand how our behavior affects the world’s
ecology, how our behavior affects other people, and studying how we
can shift attitudes and shift behavior using virtual reality
technology. The lab is really one of the pioneering research groups
in the field of VR and behavior change. And I encourage you and your
listeners to check out the website. There’s a lot of really amazing
research material that’s been accumulated there. But my role is sort
of translating that research and the research of other groups from
the academic arena out into the medical product arena. So in addition
to my work at Stanford and at the University of Colorado National
Mental Health Innovation Center, I’m doing a lot of work advising
some of the early stage and some later stage medical VR startups,
those that have received investment funding to build and bring to
market products. Then also helping those groups connect with the
healthcare ecosystem, the pharmaceutical industry, the medical device
industry, the health services industry, and the insurance providers,
making sure that everyone in the healthcare ecosystem is aware of the
power of VR and AR technology and how it can make a big difference in
healthcare.
And as you mentioned earlier, it really
spans the spectrum. We see some really amazing products are being
developed to in the education arena to help train people to not just
do surgical procedures, but also to work as a team on difficult
problem, how to deliver distressing news to a family or to a patient,
how to interview a patient in an effective way. A large selection of
training applications. But beyond training, we also have a whole new
wave of systems — like you described, for strabismus — that help
therapeutically with clinical problems. And those range from– well,
you listed a few of them, treating stroke and traumatic brain injury,
treating anxiety and depression, post-traumatic stress disorder,
helping with addictions, helping with autism. There’s a very long
list of interventions. And one of the things I’m really excited about
is VR allows us also to do a better job of assessing people, to
measure how they move, measure their mood and their behavior in ways
that we really didn’t have tools for before. We can now do better
objective assessments, instead of subjective measurements. And that
gives us some very powerful tools.
Alan: One of the tools that I
think we’re just starting to see come online is eye tracking and
motion tracking, where we’re really able to get data points about
humans that we’ve never had before. Is that some of the things you’re
working on?
Walter: Absolutely. And what I
find very exciting is that because we can measure how people move,
because we can measure where they’re looking, what they’re paying
attention to, things that before we’re collected in a subjective and
analog way, we now are able to collect in a reproducible and
objective way, and that gives us both new tools for research, but
also new tools for assessment. Let me give you a few examples. If
we’re trying to understand, let’s say, a neurodegenerative disease
like Alzheimer’s or Lewy body disease or perhaps Parkinson’s. And we
are asking people subjectively — or their family members — to
subjectively give a report on how someone’s doing cognitively. It’s
very, very hard to measure. And that means that developing new
pharmaceutical interventions, new behavioral therapy interventions
are all limited by the fact that we don’t have very precise tools.
But if we can measure how people move, if we can measure what people
are looking at, if we can measure what people are attending to, if we
can measure behavior, then we can do a better job of coming up with
new interventions that — either pharmaceutical or through cognitive
behavioral therapy — that can help with whatever problem is that
they’re dealing with. And it’s really putting us in a better position
to move forward in terms of both research and product development.
Alan: It’s incredible. The work
you were doing 30 years ago, how has it progressed? How has it
changed, from where you were starting out in this technology, to
where it is today, and where you think it will go in the next five to
ten years?
Walter: Boy, what a big
question. In terms of the progress we’ve made, we knew back decades
ago that VR could be an effective tool to help with some very
difficult problems, for example, treating phobias or post-traumatic
stress. We can use a simulation to help do what’s called exposure
therapy. So, for example, someone who might have a fear of flying, we
can develop a simulator that allows him to go through the experience,
or fear of heights, or fear of spiders, or really anything that there
is a fear reaction to. The counselor or commission can gradually, in
a controlled manner, expose a patient to what they are afraid of, and
teach them the skills to manage those fears and to habituate
sometimes what’s a learned fear response. And we were able to do
that, for example, with– there was a– Larry Hodges and Barbara
Rothbaum developed a virtual Vietnam — back in the early 90s — that
was very effective at treating post-traumatic stress disorder for
Vietnam vets, some of who had been suffering from PTSD for decades.
And with the help of this method of exposure therapy, it was able to
make a big difference.
So to answer your question, we’ve known
for a long time that we can treat problems like addiction, problems
like post-traumatic stress, problems like phobias, a whole selection
of clinical problems using VR, but it really wasn’t affordable. And
it also wasn’t very comfortable to wear. Spending a lot of time in VR
would sometimes cause simulator sickness. And it was so expensive,
and a head-mounted display could cost $70,000. A computer that was
used in the research could cost four or five hundred thousand
dollars. But now we have better systems that I can order on Amazon.
It’s just amazing. What’s happened is some of the paths that were
plowed back in the early days showing what works, what doesn’t work,
what’s a fruitful path of endeavor, what’s not. We know that. And we
can go deeper now with the technology. We can do larger scale
studies. We can reproduce the original research that was done with
small sample sizes and with what appears now to be very crude
equipment. We can really go out and build it up better and probably
more importantly, get it to people who need it.
Alan: So we’ve seen a reduction
in the cost, reduction in the time to make these things. The one
thing that we’re not seeing — and maybe you can speak to this — is
a massive adoption across all the industries. And medical is really
adopting this technology more so than in the other one. What do you
think are still the underlying reasons for the hesitation? And it
might– I have a theory on this. And my theory is that we’ve been
crying wolf for so long. “VR is going to be great!” Another
five years, “VR is going to be great!” Another five years.
People are just like “Yeah, yeah, whatever. VR, sure.” How
do we break through that?
Walter: I think that’s part of
it. I think part of it also has been that VR is somewhat of a glib
phrase. I think it’s a very descriptive phrase. But it’s also a
phrase that causes some people to think, as you described, that it
maybe it’s a little too light-duty and not an effective tool for
addressing large problems. But it’s changed. We now say VR or AR, and
people know what we mean, much like when we say AI. Whereas five
years ago, if somebody said VR, you wouldn’t necessarily know what
that stood for. So we’re getting used to that phrase. And in my
opinion, what the problem is — yes, what you described — but also,
it’s the fact that it’s sort of what we call a virality factor of
one. You almost have to see it and try it before you really get it.
And that means it’s not as contagious as some other things that catch
fire really fast. And I think it is going to really take off very
fast. But what we have to do is build the practical applications for
the enterprise. We’re in the process of doing that in architecture
and finance and skilled labor force training, soft skill training and
medicine. Now, medicine in particular, though, has a little bit of a
very appropriate barrier. We need to show what’s effective and we
need to show what’s safe. It takes a little bit time to do the
studies to demonstrate safety and efficacy, but we’re in the process
of doing that. And there’s a lot of early adopters out there who have
brought VR into their clinics, into their hospitals, and are doing
very good work with it. So I think things will catch fire very soon.
But I think we’re on schedule, now that the prices are reasonable.
And what’s slowing things down, I think, is we just need more people
to build out the practical tools that can be used in the enterprise.
Alan: So there is some massive
opportunities for enterprising students coming out of universities
like Stanford who are– I think they’re kind of benefiting from the
three decades of work that you and your colleagues have put in,
because in my opinion, what I’m seeing in the market is that now it
seems to be just ripe for explosion. It looks like we’ve got, the
rocket is on the track and all thrusters are going. And is that the
kind of sentiment that you’re feeling in the market right now?
Walter: I think it’s a good
analogy. I think that because the groundwork has been laid and
because the infrastructure is already in place. VR and AR technology
leverages the Internet and broadband that’s in place. 5G is on its
way. We’re leveraging distribution mechanisms. And also keep in mind
that at least in the medical arena, a big part of what we’re doing
relies upon other technologies such as machine learning. I mean, we
collect a lot data, but we need to have the tools to analyze it. It
relies upon avatar technology that look much more realistic and have
facial expressions and nonverbal communication aspects to them, so
that they look much more realistic than what we’ve been able to
generate in the past. So I think things are poised to take off
because there’s a convergence of technology. AR and VR technology is
going to leverage AI technology, it’s going to leverage simulation
technology that’s being used in a variety of other arenas other than
healthcare. And it’s all converging into one spot. So, yes, I think
it is poised– and I use the term poised, but I don’t mean poised for
taking off in ten years, and I don’t mean even five years. I think it
is in the process taking off now, and it’s going to move really fast.
Alan: I couldn’t agree more, to
be honest. I’ve been studying this industry inside and out and I
subscribe to Google Alerts for virtual and augmented reality. And
five years ago, I’d maybe get one alert every two days and have a
couple of things in it. I’m getting three alerts a day and they’re
packed. Let me read something. I think this is interesting:
“Medical practitioners should suit
the virtual reality application to the patient, not the patient to
the technology. The VR technology agenda in medicine. All
organizations face the problem of dramatic increases in volume of
data that they must manage to conduct their daily affairs.
Increasingly, this data is in a variety of media formats,
particularly in medicine, where data formats include CAT, MRI, EEG
and X-ray images, as well as real time communication with
consultants. Many technologies have been offered over the past 40
years to help with this escalating information resource management
problem. And now we have virtual reality.”
That is from Virtual Reality
Magazine, from 1993.
Walter: [laughs] That doesn’t
surprise me.
Alan: [laughs] I happen to have
three copies of this magazine. It’s just mindblowing that the promise
was there and it was 20 years too early. And now with the advent of
Oculus Quest and these really inexpensive headsets, I think it’s just
opened up the world to developers, and clinicians, and medical
practitioners.
Walter: Well, let me mention, to
snap on top of that point. I think there’s another trend that’s
worthy of paying attention to which is, at least in medicine, the
continents are colliding. The consumer electronics companies like
Apple and Samsung and to some extent Google and Microsoft, along with
companies like Amazon, are jumping nto healthcare. And that’s really
changing the game. The fact that with their speed of product
development and they’re savvy about good user interface design, to
have these groups jump in and team up with pharmaceutical companies
like Novartis and Sunovion and others, and to team up with medical
device companies like Penumbra, it’s really a wonderful time to see
the confluence of speed and savvy that is coming from the consumer
electronics company, combined with the experience of the medical
product development and distribution channel. It’s like a tidal wave
roaring down a racetrack. So I think things are going take off faster
than any of us will expect.
Alan: I’m going to read
something else. This is from a different magazine:
“Virtual Interface Technology
offers many applications for assisting disabled persons: augmenting
reality in rehabilitation medicine.”
By Walter J. Greenleaf and Maria A.
Tovar, from Spring 1995, Virtual Reality Magazine Special
Report.
Walter: Uh-huh. That’s right,
that’s right. And finally, the things that we were excited about back
then are now affordable. We’ve been able to do it in research
laboratories for decades. But now we can start moving it out into
clinical care.
Alan: So, people listening…
let’s say, for example, hospitals or clinicians or– what is the
practical first steps for them to start applying these technologies?
What would you recommend? I know you founded the IVRHA, so maybe you
can talk about that and how these hospitals and clinicians and
medical practitioners can– how can they get into this and start
using it right away?
Walter: Well, first of all, I’m
glad you asked about practical applications as opposed to asking for
“the killer app,” because we don’t use that phrase in medicine.
You’d be amazed at how many startups coming from the technology.
Alan: [laughs] Probably the
worst phrase ever for medicine. [laughs]
Walter: Yeah, but you know,
people still are talking about the killer app for medicine. But to
talk about how people can get started, I think going to the IVRHA
website is a good spot. You’ll see many of the medical product
companies and startups and service providers that are teaming to
explore the applications of the technology, it’s a great, great
starting spot. We also put on a yearly conference on medical VR and
that’s a great spot to come and meet other people that are working
the arena and try out demos.
Alan: Where’s that going to be?
Walter: It’s going to be in
Nashville. There’s a link to the website for it at the IVRHA website.
I think another way really for clinical groups to get started is to
well, I think take a look at some of the startups out there that are
doing pioneering work. You described how VR is being used to help
treat strabismus and amblyopia. For example, I would think for really
whatever indication that you’re interested in, be it depression, be
it anxiety, be it stroke rehabilitation, be it pain distraction, a
good way to get services, to just, of course, do a simple web search
to find the principal groups that are out there doing research, and
then see which product development companies are basing their
products on a research backed initiative and see who is citing and
using as part of their advisory group established research groups.
There are a lot of people that are jumping into the clinical VR arena
by translating their skills in game development or their skills in
sensor development into products. But not all of them understand the
medical ecosystem. Not all of them understand the ergonomics of how
to bring a product into a clinical environment, so that it doesn’t
slow down the process, so it doesn’t create additional burden and
work for the clinicians and the ancillary care staff.
So I would say as my filter, I would
look for those product development companies that have either brought
onboard research scientists and experts who know the medical
ecosystem to help them advise their technology direction and probably
more importantly, conduct validation studies. And I would also look
for those that have formed alliances with some of the existing
experts in the medical ecosystem, like some of the pharmaceutical
companies or medical device companies. They’re very selective. And of
course, the ultimate criteria is if somebody has made the effort to
go and receive FDA certification, has a product to demonstrate their
safety and their efficacy. That’s really the main watchword I would
look forward to, to see where to start is to find those companies
that address the problem you’re interested in that have gone down
that pathway.
Alan: Absolutely. You touched on
a couple of use cases, one of them being pain distraction. One of the
studies I read, it was something like 25 percent reduction in opioid
usage in debridement of wounds. And if you think about that, that’s a
massive reduction in medications that have detrimental effects on
people, and can lead to addictions and other things, just by using
VR.
Walter: It is very exciting.
You’re you’re talking about Hunter Hoffman’s work. And Hunter has
done a great job of demonstrating not only how we can use VR to
reduce the need for narcotics in a burn treatment clinic, for
example. But how the reduction in the need for narcotics translates
out into less possibility for addiction after people are discharged.
And that’s a really significant thing. A large proportion of the
problems we have with opiate addiction in our country right now are
because people get caught up in using narcotics as a result of being
in a hospital and appropriately getting pain reduction medication.
But if we can augment the pain medication and reduce the need for it
by using virtual environments as a gating distraction from the pain,
then all the better. And also VR been used for helping people with
chronic pain learn through cognitive behavioral therapy and other
approaches how to manage the chronic pain. So it’s not just during
the acute phase of a painful process, but also for the post
discharge, post acute phase.
Alan: When people are thinking
about this, what kind of, in business terms we’d say key performance
indicators. How do you measure the success of something over another
modality? What are the typical measurements if you’re going to start
using VR versus something else used in your clinic? How would you
measure that success? What does that look like?
Walter: That’s a very good
question. I would say for a medical product, it really depends if
we’re teaching a procedure that’s very different than if we have a
clinical assessment, and that’s very different than if we have a
clinical intervention. They all have different metrics. If you’re
measuring the ability to use VR to improve a training process, one
thing you would look at is not just how much more proficient the
training process is in terms of mastering a skill — such as maybe a
surgical skill or how to diagnose a patient — but you also would
want to know comparing it to the traditional methods. Is it less
expensive? By and large, it would be. Often medical schools have to
employ actors and actresses to help in part of the training process
or use very expensive simulating machines. You’d also want to know
what the retention is. Do people retain the lessons learned in a more
experiental matter with VR compared to other ways of learning, such
as looking at a videotape or reading a textbook or dissecting a
cadaver? And by the way, there are some medical schools that are
switching over to all electronic cadavers now, which is not only less
expensive, but in many ways more dynamic, they can overlay the image
of the cadaver with extra information and have a more structured way
of training. So for training, we would have those metrics.
For diagnosis, it would be are we able
to do a less expensive, more efficient, more accurate diagnosis to a
better differential diagnosis, maybe come up with less false
positives and false negatives as a diagnostic criteria? We would use
that. And also in all these things, we need to look at the cost. Does
the extra cost of deploying a VR system– and not just the cost of
buying the equipment, but the cost of putting it into the hospital
ecosystem, maintaining it, supporting it. Maybe there’s a need for
extra personnel to keep the batteries charged. So you really have to
look at the whole effect of bringing new technology into the medical
ecosystem. So that needs to be part of it. And same thing when for
when we use VR as a clinical intervention, does it save us money?
Does it save in the long run? Does it produce better healthcare
results? And you know, I have to tell you, Alan, when I’ve looked at
the metrics on all three of these things, training, assessments and
interventions, VR really can make a big difference. It really is cost
effective. But again, it has to be positioned the right way and not
brought into the clinical ecosystem in such a way that it produces an
undue burden. You want the clinicians to be able go home earlier, not
have to say a little bit later to do more paperwork.
Alan: It’s interesting you say
that because one of the things that came up at LiveWorx was device
management. We’re used to managing cell phones and iPads and stuff,
but this adds a whole new element of device management. It’s
something that people don’t consider until they have 50 of them to
deal with, or 500 of them to deal with.
Walter: Exactly the issue. But I
think the savvy product developers will appreciate that and that they
are building into their systems ways to reduce the paperwork burden,
improve the workflow of the clinic along with having a new tool.
Alan: It’s worth the investment,
in my opinion. From the numbers that you’re seeing, what are some of
the expected outcomes? Let’s say, for example, using this for autism,
it’s very hard to say with a number that this was more efficient or
something. It’s almost anecdotal, but there’s got to be numbers there
that prove that this technology far outpaces anything else we’ve ever
created.
Walter: Well, part of the way we
do that in medicine is by doing long term follow up. If, for example,
we use virtual reality to help with stroke rehabilitation, both
during the acute phase after a stroke and then the post acute, when
people are coming into a clinic for follow-up or doing home
rehabilitation, if we look at how they’re doing a few years later and
compare that to people who had standard treatment, if there’s less
permanent disability, if people are returning to work sooner, if
people are able to be more functional and take care of activities —
daily living is the phrase that we use — more effectively after
they’ve used VR and AR enabled interventions as opposed to the
standard, then that tells us something and we can put a number on
that in terms of dollars saved by preventing permanent disability and
helping people get back to work sooner. Same thing for– well, let’s
use the example of treating someone who has fear of heights or fear
of flying. We can compare those people who used a VR system to master
those problems and see how they’re doing a few years later. How many
of them are still flying? How many of them need to use maybe
anti-anxiety drug in order to get on a plane, versus those who don’t?
If the ones who had the VR treatment are doing better, then that
tells us a lot and we can quantify the value of that.
Alan: Let me ask you another
question from a different angle. Have you ever seen a time where VR
wasn’t more impactful or better than the traditional way of doing
things?
Walter: That’s a good question,
Alan. We’re still in a phase where we’re mapping VR to the problems,
and I think– Nothing is really coming to mind. I’m sure there have
been situations where we thought that VR would be a better way of
approaching this problem, and it turned out to not really be cost
effective. But I have to say, I’m really at a loss to come up with a
good example. I think mostly because, again, for many years the
technology was so expensive, we were very careful to really think
through how to apply the technology the right way. Now, I should say
I’ve seen a lot of where I think it’s not necessarily always the best
use. A lot of VR technology to help people learn to relax. And some
of those VR environments I think are not that much different than
listening to an audio tape or just closing your eyes and imagining
that you’re sitting at a peaceful spot. I think that there are very
amazing mindfulness training programs that promote an active
engagement, where you’re not just at a beach listening to the waves.
And those are very exciting. But I think there are some where the
incremental value of just having a VR environment is not as amazing
as it could be, but it’s still a learning phase. We’re learning how
to leverage the technology in the best way.
Alan: One of the things I saw
was — and I think it was the Cubicle Ninjas VR meditation, it might
have been them or another one — you actually connected your
smartwatch to the VR headset so it could read your pulse. You were
able to go through these deep breathing exercises guided, it is like
a guided meditation, deep breathing and you could actually watch your
heart rate go down as you controlled your breathing.
Walter: That is a good example
of a good use case, where we’re leveraging not just the display
technology, but incorporating the measurement of psycho-physiological
signals and using it to dynamically change the environment. So like
anything, there’s a spectrum of innovation that can be applied. And I
think when you asked me where VR isn’t really that effective, all I’m
really finding is to think of our times where, it’s effective but not
as effective as it could be. If we add extra layers of analytics, or
extra layers of social connection, or extra layers of leveraging game
technology to make things more rewarding and exciting. So but again,
we’re really just getting started now. Now that the products are more
affordable, I think you’ll see a surge of people applying them to
clinical issues. And we probably will see some that are sort of
poorly designed and lame, but I’m sure we’re going to see some that
are just incredible, too.
Alan: Yeah. I would have to say
that we’ve kind of passed the point where we know which VR techniques
make people sick. So I get really, really bad motion sickness.
Walter: Uh-huh.
Alan: I’m the guinea pig for the
office. But yeah, there are still some companies out there making
things that make people sick.
Walter: Yeah, and I think that’s
a matter of poor design, if you moved the world around someone, it’s
bound to make them sick. But if you give them agency to move through
the world, it’s very different. Let me tell you about one way we’re
applaying VR and AR technology to medicine that I’m particularly
excited about. There’s a focus now in medicine to move towards more
what we call precision medicine, where instead of a one-size-fits-all
approach for treatments — where you go down a clinical pathway and
say, well, let’s try this and see if it works, and if it doesn’t
work, we’ll try something else — where we leverage the genomic
information about individual, the measurements we can get about their
behavior and their physiology. And we challenge them with virtual
environments, for example, to see how they react. And we come up with
a precision pathway for treatment that is based on their specific
biotype.
One of the projects we are in the midst
of doing at Stanford University is a project headed up by B Williams
called the Engage Project, where we’re looking at people who have
comorbid depression and weight management issues. And we’re following
them over a multi-year period of time, as they learn to manage their
weight and manage the moods. And they have a variety of different
interventions. But as they go down that treatment pathway, we’re
imaging their brains using a functional medical imaging device. And
we’re also measuring their behavior using data collected by– they
opt-in to allowing us to collect data on their smartphones about how
fast they’re typing, and how fast they’re swiping, and their general
activity levels. And then we’re also using virtual reality as a way
of challenging their neural circuits to see– we have a challenge,
much like Beat Saber, where they have to learn to react the right way
and maybe some other times control their reaction. And the idea here
is to come up with a VR based system that can be used to identify
different neuro circuit biotypes, and that can be used to come up
with a more precise approach for treating depression, for example. So
for biotype A, we might recommend that they try this treatment for
depression, and a different biotype — again, identified by how they
react to a stimulus in a virtual environment — might be recommended
to go down treatment pathway B and a third person might go down
treatment pathway C, all based on their assessment that we’re able to
do using virtual reality. So it’s a research project, but it harkens
to the vision that we can leverage not just VR technology, but also
sensor technology, biosensor technology, and machine learning
technology to really come up with a better way of adapting our
medical interventions to the specific individual.
Alan: It’s incredible. You talk
about the three pillars: training, assessment and intervention. But
one of the things that I think must be under the assessment side of
things is being able to visualize your data. As a physician, being
able to take an MRI data and blow it up and get right in there, I
think we’re only scratching the surface of that. Have you seen any
real world applications of visualizing medical data in unique ways in
VR and AR?
Walter: Oh, absolutely. For
example, there’s some groups doing virtual colonoscopies where they
collect the colonoscopy data, but sometimes it collected just using
an imaging machine as opposed to doing the actual physical
colonoscopy and then allows a clinician sort of fly through the colon
looking for polyps. Or same thing for pre-surgical planning. We can
take CAT scan data, fuse it with ultrasound data, fuse it with other
data, and then plan a complex surgical procedure in advance. So
allowing the clinicians to– not just the radiologist but also the
surgeons, to visualize in three dimensions the complex process that
they need to go through in their operation and rehearse it in
advance.
I’d like to add to those pillars that
you mentioned, too, though. It’s not just training and assessments
and interventions. I think the AR technology is also going to be very
helpful for promoting health and wellness, showing us the effect of
our behavior and promoting us to do what we really should do to be
healthy. It’s a very difficult thing to get yourself to exercise, get
yourself to eat the right way, to remember to take your medications,
especially if they have maybe side effects that you don’t like, and
to understand the consequences of not adhering to what you should be
doing to keep yourself healthy. One technique that was pioneered at
the Stanford Virtual Human Interaction Lab was to create an image of
your future self, an avatar of your future self that you could talk
to and who could talk to you, and show you in a shortened timeframe
the consequences of your decision. So if, for example, your
smartphone is sensing that you’re going into a bar and you’ve already
decided that really you want to cut back on your alcohol use, your
phone can ring, and there on your phone is your future self. Someone
who looks like you, sounds like you, but 20 years older saying, “Hey,
I thought we talked about this. Look at what you’re doing to me.”
Or maybe if you’re not exercising enough or if you’re not eating the
right way, it could show up in your future self looking very
unhealthy. Same problem for managing addictions.
So I think prevention wellness is
another pillar. I think another thing to keep in mind is I think VR
and AR technology is going to allow us to reach underserved
populations. I think the focus of care will shift from the clinic to
wherever the individual is. There’s some things, of course, we have
to do at a specialized treatment center where there’s specialized
equipment. But I think for a lot of things, especially in terms of
behavioral medicine and psychology and psychiatry, I think we’ll be
able to start doing a lot more reaching underserved populations and
be able to provide effective treatments by leveraging the
telemedicine nature of VR and AR technology.
Alan: Well, that’s wonderful.
It’s a nice close to this conversation, because if we can use these
technologies to democratize wellness and health, not just medicine,
not treating people when they’re sick, but really just keeping people
healthy, I think that is the fundamental shift that we need to make
as a society. And I think VR and AR lends a very good hand to that.
Walter: Well, and especially if
we can follow the pathways that our colleagues in the computer gaming
arena have pioneered. They know how to grab people’s attention and
keep them involved in a process. So if we can look at how they have
learned to do that in the gaming arena and apply that to some of the
interventions we’re doing in the medical arena, I think we can make
medicine not just more effective, but also more engaging and more
palatable.
Alan: And fun. Let’s not take
away the fun. I want to say, on behalf of all the listeners and
myself included. Thank you so much, Dr. Greenleaf, for taking the
time to be on this podcast. It’s been really amazing.
Walter: Well, thank you, Alan.
Thanks for all the good work you do. Getting the word out and for the
good questions today.
View all episodes
By Alan Smithson from MetaVRse
4.5
1212 ratings
XR has come a long way, baby – and we have one of the technology’s earliest pioneer’s on today’s episode. Dr. Walter Greenleaf has been working in the field for 33 years, since the days when VR was little more than a twinkle in research scientists’ eyes. Now, he and Alan chat about how far the technology has come, and how far it still has to go.
Alan: Welcome to the XR for
Business Podcast with your host, Alan Smithson. Today’s guest is Dr.
Walter Greenleaf, a behavioral neuroscientist and medical technology
developer working at Stanford University. With over three decades of
research and development experience in the field of digital medicine
and medical virtual reality technology, Walter is considered the
leading authority in the field of working in this industry, and he’s
been doing this for 33 years. Unbelievable experience. Dr. Greenleaf
has designed and developed numerous clinical systems over the last 33
years, including products in the fields of surgical simulation, 3D
medical visualization, telerehabilitation, clinical informatics,
clinical decision support, point of care, clinical data collection,
ergonomic evaluation technology, automatic sleep staging systems,
psycho-physiological assessment and simulation assisted
rehabilitation technologies, as well as products for behavioral
medicine. Dr. Greenleaf’s focus has always been on computer supported
clinical products, with a specific focus on virtual reality and
digital health technologies to treat post-traumatic stress disorder,
anxiety disorders, traumatic brain injury, stroke, addictions, autism
and other difficult problems, and behavioral and physical medicine.
He’s currently a distinguished visiting scholar at Stanford
University’s Media X program at Stanford University’s Virtual Human
Interaction Lab and the Director of Technology Strategy at the
University of Colorado National Mental Health Institute Center. To
learn more about the work that Dr. Greenleaf and his team are doing,
you can visit the Human Interaction Lab at Stanford at
vhil.stanford.edu and a new
organization that he’s formed called the International VR Health
Association at ivrha.org.
Welcome to the show, Dr. Walter
Greenleaf. So great to have you.
Walter: Thanks, Alan. I’m
pleased to be here with you.
Alan: That’s an honor. You are
considered one of the godfathers of this technology. You’ve been
working in it your whole life. And I want to personally say thank you
for laying the groundwork that allows people like myself — the new
people getting involved here — to really pick up where you left off,
and where you’ve driven this whole industry forward to, and let us
really build upon your lifetimes of knowledge. So thank you very much
for paving the way for us.
Walter: Well, thank you, Alan.
And really, without everybody else’s and my colleagues work and your
work and other people who are helping the trends position from
something that for a long time was a research lab curiosity and
something that really hadn’t escaped the confines of academia. Now we
have it out there in the world. And I’m particularly excited about
all the progress has been made in applying VR and AR technology to
difficult problems in healthcare. For me, it’s a very exciting time.
Alan: I’ve been keeping track of
all of the different things that come up in my news feed and I have a
health and medical folder. And it’s interesting, because last year I
actually had to break it apart into a mental health folder, in
addition to the traditional health and medical. So there is an
enormous amount of, not just research, but real practical
applications being created for this. You know, one of them, one that
stands out the most to me is being able to use virtual reality to
treat lazy eye or strabismus. I thought that was just amazing. Within
a few sessions, people are seeing complete reduction and elimination
of their lazy eye using virtual reality. And that’s just one of a
thousand use cases in this technology. So maybe walk us through the
work you’re doing at the Virtual Human Interaction Lab, and what are
the some of the great use cases that you’re seeing now?
Walter: OK, well, sure. Jeremy
Bailenson is the director of the Stanford Virtual Human Interaction
Lab, and I help out there as the medical advisor and expert. The
Stanford Virtual Human Interaction Lab has a focus on exploring and
studying how VR technology can promote pro-social behavior, such as
helping us better understand how our behavior affects the world’s
ecology, how our behavior affects other people, and studying how we
can shift attitudes and shift behavior using virtual reality
technology. The lab is really one of the pioneering research groups
in the field of VR and behavior change. And I encourage you and your
listeners to check out the website. There’s a lot of really amazing
research material that’s been accumulated there. But my role is sort
of translating that research and the research of other groups from
the academic arena out into the medical product arena. So in addition
to my work at Stanford and at the University of Colorado National
Mental Health Innovation Center, I’m doing a lot of work advising
some of the early stage and some later stage medical VR startups,
those that have received investment funding to build and bring to
market products. Then also helping those groups connect with the
healthcare ecosystem, the pharmaceutical industry, the medical device
industry, the health services industry, and the insurance providers,
making sure that everyone in the healthcare ecosystem is aware of the
power of VR and AR technology and how it can make a big difference in
healthcare.
And as you mentioned earlier, it really
spans the spectrum. We see some really amazing products are being
developed to in the education arena to help train people to not just
do surgical procedures, but also to work as a team on difficult
problem, how to deliver distressing news to a family or to a patient,
how to interview a patient in an effective way. A large selection of
training applications. But beyond training, we also have a whole new
wave of systems — like you described, for strabismus — that help
therapeutically with clinical problems. And those range from– well,
you listed a few of them, treating stroke and traumatic brain injury,
treating anxiety and depression, post-traumatic stress disorder,
helping with addictions, helping with autism. There’s a very long
list of interventions. And one of the things I’m really excited about
is VR allows us also to do a better job of assessing people, to
measure how they move, measure their mood and their behavior in ways
that we really didn’t have tools for before. We can now do better
objective assessments, instead of subjective measurements. And that
gives us some very powerful tools.
Alan: One of the tools that I
think we’re just starting to see come online is eye tracking and
motion tracking, where we’re really able to get data points about
humans that we’ve never had before. Is that some of the things you’re
working on?
Walter: Absolutely. And what I
find very exciting is that because we can measure how people move,
because we can measure where they’re looking, what they’re paying
attention to, things that before we’re collected in a subjective and
analog way, we now are able to collect in a reproducible and
objective way, and that gives us both new tools for research, but
also new tools for assessment. Let me give you a few examples. If
we’re trying to understand, let’s say, a neurodegenerative disease
like Alzheimer’s or Lewy body disease or perhaps Parkinson’s. And we
are asking people subjectively — or their family members — to
subjectively give a report on how someone’s doing cognitively. It’s
very, very hard to measure. And that means that developing new
pharmaceutical interventions, new behavioral therapy interventions
are all limited by the fact that we don’t have very precise tools.
But if we can measure how people move, if we can measure what people
are looking at, if we can measure what people are attending to, if we
can measure behavior, then we can do a better job of coming up with
new interventions that — either pharmaceutical or through cognitive
behavioral therapy — that can help with whatever problem is that
they’re dealing with. And it’s really putting us in a better position
to move forward in terms of both research and product development.
Alan: It’s incredible. The work
you were doing 30 years ago, how has it progressed? How has it
changed, from where you were starting out in this technology, to
where it is today, and where you think it will go in the next five to
ten years?
Walter: Boy, what a big
question. In terms of the progress we’ve made, we knew back decades
ago that VR could be an effective tool to help with some very
difficult problems, for example, treating phobias or post-traumatic
stress. We can use a simulation to help do what’s called exposure
therapy. So, for example, someone who might have a fear of flying, we
can develop a simulator that allows him to go through the experience,
or fear of heights, or fear of spiders, or really anything that there
is a fear reaction to. The counselor or commission can gradually, in
a controlled manner, expose a patient to what they are afraid of, and
teach them the skills to manage those fears and to habituate
sometimes what’s a learned fear response. And we were able to do
that, for example, with– there was a– Larry Hodges and Barbara
Rothbaum developed a virtual Vietnam — back in the early 90s — that
was very effective at treating post-traumatic stress disorder for
Vietnam vets, some of who had been suffering from PTSD for decades.
And with the help of this method of exposure therapy, it was able to
make a big difference.
So to answer your question, we’ve known
for a long time that we can treat problems like addiction, problems
like post-traumatic stress, problems like phobias, a whole selection
of clinical problems using VR, but it really wasn’t affordable. And
it also wasn’t very comfortable to wear. Spending a lot of time in VR
would sometimes cause simulator sickness. And it was so expensive,
and a head-mounted display could cost $70,000. A computer that was
used in the research could cost four or five hundred thousand
dollars. But now we have better systems that I can order on Amazon.
It’s just amazing. What’s happened is some of the paths that were
plowed back in the early days showing what works, what doesn’t work,
what’s a fruitful path of endeavor, what’s not. We know that. And we
can go deeper now with the technology. We can do larger scale
studies. We can reproduce the original research that was done with
small sample sizes and with what appears now to be very crude
equipment. We can really go out and build it up better and probably
more importantly, get it to people who need it.
Alan: So we’ve seen a reduction
in the cost, reduction in the time to make these things. The one
thing that we’re not seeing — and maybe you can speak to this — is
a massive adoption across all the industries. And medical is really
adopting this technology more so than in the other one. What do you
think are still the underlying reasons for the hesitation? And it
might– I have a theory on this. And my theory is that we’ve been
crying wolf for so long. “VR is going to be great!” Another
five years, “VR is going to be great!” Another five years.
People are just like “Yeah, yeah, whatever. VR, sure.” How
do we break through that?
Walter: I think that’s part of
it. I think part of it also has been that VR is somewhat of a glib
phrase. I think it’s a very descriptive phrase. But it’s also a
phrase that causes some people to think, as you described, that it
maybe it’s a little too light-duty and not an effective tool for
addressing large problems. But it’s changed. We now say VR or AR, and
people know what we mean, much like when we say AI. Whereas five
years ago, if somebody said VR, you wouldn’t necessarily know what
that stood for. So we’re getting used to that phrase. And in my
opinion, what the problem is — yes, what you described — but also,
it’s the fact that it’s sort of what we call a virality factor of
one. You almost have to see it and try it before you really get it.
And that means it’s not as contagious as some other things that catch
fire really fast. And I think it is going to really take off very
fast. But what we have to do is build the practical applications for
the enterprise. We’re in the process of doing that in architecture
and finance and skilled labor force training, soft skill training and
medicine. Now, medicine in particular, though, has a little bit of a
very appropriate barrier. We need to show what’s effective and we
need to show what’s safe. It takes a little bit time to do the
studies to demonstrate safety and efficacy, but we’re in the process
of doing that. And there’s a lot of early adopters out there who have
brought VR into their clinics, into their hospitals, and are doing
very good work with it. So I think things will catch fire very soon.
But I think we’re on schedule, now that the prices are reasonable.
And what’s slowing things down, I think, is we just need more people
to build out the practical tools that can be used in the enterprise.
Alan: So there is some massive
opportunities for enterprising students coming out of universities
like Stanford who are– I think they’re kind of benefiting from the
three decades of work that you and your colleagues have put in,
because in my opinion, what I’m seeing in the market is that now it
seems to be just ripe for explosion. It looks like we’ve got, the
rocket is on the track and all thrusters are going. And is that the
kind of sentiment that you’re feeling in the market right now?
Walter: I think it’s a good
analogy. I think that because the groundwork has been laid and
because the infrastructure is already in place. VR and AR technology
leverages the Internet and broadband that’s in place. 5G is on its
way. We’re leveraging distribution mechanisms. And also keep in mind
that at least in the medical arena, a big part of what we’re doing
relies upon other technologies such as machine learning. I mean, we
collect a lot data, but we need to have the tools to analyze it. It
relies upon avatar technology that look much more realistic and have
facial expressions and nonverbal communication aspects to them, so
that they look much more realistic than what we’ve been able to
generate in the past. So I think things are poised to take off
because there’s a convergence of technology. AR and VR technology is
going to leverage AI technology, it’s going to leverage simulation
technology that’s being used in a variety of other arenas other than
healthcare. And it’s all converging into one spot. So, yes, I think
it is poised– and I use the term poised, but I don’t mean poised for
taking off in ten years, and I don’t mean even five years. I think it
is in the process taking off now, and it’s going to move really fast.
Alan: I couldn’t agree more, to
be honest. I’ve been studying this industry inside and out and I
subscribe to Google Alerts for virtual and augmented reality. And
five years ago, I’d maybe get one alert every two days and have a
couple of things in it. I’m getting three alerts a day and they’re
packed. Let me read something. I think this is interesting:
“Medical practitioners should suit
the virtual reality application to the patient, not the patient to
the technology. The VR technology agenda in medicine. All
organizations face the problem of dramatic increases in volume of
data that they must manage to conduct their daily affairs.
Increasingly, this data is in a variety of media formats,
particularly in medicine, where data formats include CAT, MRI, EEG
and X-ray images, as well as real time communication with
consultants. Many technologies have been offered over the past 40
years to help with this escalating information resource management
problem. And now we have virtual reality.”
That is from Virtual Reality
Magazine, from 1993.
Walter: [laughs] That doesn’t
surprise me.
Alan: [laughs] I happen to have
three copies of this magazine. It’s just mindblowing that the promise
was there and it was 20 years too early. And now with the advent of
Oculus Quest and these really inexpensive headsets, I think it’s just
opened up the world to developers, and clinicians, and medical
practitioners.
Walter: Well, let me mention, to
snap on top of that point. I think there’s another trend that’s
worthy of paying attention to which is, at least in medicine, the
continents are colliding. The consumer electronics companies like
Apple and Samsung and to some extent Google and Microsoft, along with
companies like Amazon, are jumping nto healthcare. And that’s really
changing the game. The fact that with their speed of product
development and they’re savvy about good user interface design, to
have these groups jump in and team up with pharmaceutical companies
like Novartis and Sunovion and others, and to team up with medical
device companies like Penumbra, it’s really a wonderful time to see
the confluence of speed and savvy that is coming from the consumer
electronics company, combined with the experience of the medical
product development and distribution channel. It’s like a tidal wave
roaring down a racetrack. So I think things are going take off faster
than any of us will expect.
Alan: I’m going to read
something else. This is from a different magazine:
“Virtual Interface Technology
offers many applications for assisting disabled persons: augmenting
reality in rehabilitation medicine.”
By Walter J. Greenleaf and Maria A.
Tovar, from Spring 1995, Virtual Reality Magazine Special
Report.
Walter: Uh-huh. That’s right,
that’s right. And finally, the things that we were excited about back
then are now affordable. We’ve been able to do it in research
laboratories for decades. But now we can start moving it out into
clinical care.
Alan: So, people listening…
let’s say, for example, hospitals or clinicians or– what is the
practical first steps for them to start applying these technologies?
What would you recommend? I know you founded the IVRHA, so maybe you
can talk about that and how these hospitals and clinicians and
medical practitioners can– how can they get into this and start
using it right away?
Walter: Well, first of all, I’m
glad you asked about practical applications as opposed to asking for
“the killer app,” because we don’t use that phrase in medicine.
You’d be amazed at how many startups coming from the technology.
Alan: [laughs] Probably the
worst phrase ever for medicine. [laughs]
Walter: Yeah, but you know,
people still are talking about the killer app for medicine. But to
talk about how people can get started, I think going to the IVRHA
website is a good spot. You’ll see many of the medical product
companies and startups and service providers that are teaming to
explore the applications of the technology, it’s a great, great
starting spot. We also put on a yearly conference on medical VR and
that’s a great spot to come and meet other people that are working
the arena and try out demos.
Alan: Where’s that going to be?
Walter: It’s going to be in
Nashville. There’s a link to the website for it at the IVRHA website.
I think another way really for clinical groups to get started is to
well, I think take a look at some of the startups out there that are
doing pioneering work. You described how VR is being used to help
treat strabismus and amblyopia. For example, I would think for really
whatever indication that you’re interested in, be it depression, be
it anxiety, be it stroke rehabilitation, be it pain distraction, a
good way to get services, to just, of course, do a simple web search
to find the principal groups that are out there doing research, and
then see which product development companies are basing their
products on a research backed initiative and see who is citing and
using as part of their advisory group established research groups.
There are a lot of people that are jumping into the clinical VR arena
by translating their skills in game development or their skills in
sensor development into products. But not all of them understand the
medical ecosystem. Not all of them understand the ergonomics of how
to bring a product into a clinical environment, so that it doesn’t
slow down the process, so it doesn’t create additional burden and
work for the clinicians and the ancillary care staff.
So I would say as my filter, I would
look for those product development companies that have either brought
onboard research scientists and experts who know the medical
ecosystem to help them advise their technology direction and probably
more importantly, conduct validation studies. And I would also look
for those that have formed alliances with some of the existing
experts in the medical ecosystem, like some of the pharmaceutical
companies or medical device companies. They’re very selective. And of
course, the ultimate criteria is if somebody has made the effort to
go and receive FDA certification, has a product to demonstrate their
safety and their efficacy. That’s really the main watchword I would
look forward to, to see where to start is to find those companies
that address the problem you’re interested in that have gone down
that pathway.
Alan: Absolutely. You touched on
a couple of use cases, one of them being pain distraction. One of the
studies I read, it was something like 25 percent reduction in opioid
usage in debridement of wounds. And if you think about that, that’s a
massive reduction in medications that have detrimental effects on
people, and can lead to addictions and other things, just by using
VR.
Walter: It is very exciting.
You’re you’re talking about Hunter Hoffman’s work. And Hunter has
done a great job of demonstrating not only how we can use VR to
reduce the need for narcotics in a burn treatment clinic, for
example. But how the reduction in the need for narcotics translates
out into less possibility for addiction after people are discharged.
And that’s a really significant thing. A large proportion of the
problems we have with opiate addiction in our country right now are
because people get caught up in using narcotics as a result of being
in a hospital and appropriately getting pain reduction medication.
But if we can augment the pain medication and reduce the need for it
by using virtual environments as a gating distraction from the pain,
then all the better. And also VR been used for helping people with
chronic pain learn through cognitive behavioral therapy and other
approaches how to manage the chronic pain. So it’s not just during
the acute phase of a painful process, but also for the post
discharge, post acute phase.
Alan: When people are thinking
about this, what kind of, in business terms we’d say key performance
indicators. How do you measure the success of something over another
modality? What are the typical measurements if you’re going to start
using VR versus something else used in your clinic? How would you
measure that success? What does that look like?
Walter: That’s a very good
question. I would say for a medical product, it really depends if
we’re teaching a procedure that’s very different than if we have a
clinical assessment, and that’s very different than if we have a
clinical intervention. They all have different metrics. If you’re
measuring the ability to use VR to improve a training process, one
thing you would look at is not just how much more proficient the
training process is in terms of mastering a skill — such as maybe a
surgical skill or how to diagnose a patient — but you also would
want to know comparing it to the traditional methods. Is it less
expensive? By and large, it would be. Often medical schools have to
employ actors and actresses to help in part of the training process
or use very expensive simulating machines. You’d also want to know
what the retention is. Do people retain the lessons learned in a more
experiental matter with VR compared to other ways of learning, such
as looking at a videotape or reading a textbook or dissecting a
cadaver? And by the way, there are some medical schools that are
switching over to all electronic cadavers now, which is not only less
expensive, but in many ways more dynamic, they can overlay the image
of the cadaver with extra information and have a more structured way
of training. So for training, we would have those metrics.
For diagnosis, it would be are we able
to do a less expensive, more efficient, more accurate diagnosis to a
better differential diagnosis, maybe come up with less false
positives and false negatives as a diagnostic criteria? We would use
that. And also in all these things, we need to look at the cost. Does
the extra cost of deploying a VR system– and not just the cost of
buying the equipment, but the cost of putting it into the hospital
ecosystem, maintaining it, supporting it. Maybe there’s a need for
extra personnel to keep the batteries charged. So you really have to
look at the whole effect of bringing new technology into the medical
ecosystem. So that needs to be part of it. And same thing when for
when we use VR as a clinical intervention, does it save us money?
Does it save in the long run? Does it produce better healthcare
results? And you know, I have to tell you, Alan, when I’ve looked at
the metrics on all three of these things, training, assessments and
interventions, VR really can make a big difference. It really is cost
effective. But again, it has to be positioned the right way and not
brought into the clinical ecosystem in such a way that it produces an
undue burden. You want the clinicians to be able go home earlier, not
have to say a little bit later to do more paperwork.
Alan: It’s interesting you say
that because one of the things that came up at LiveWorx was device
management. We’re used to managing cell phones and iPads and stuff,
but this adds a whole new element of device management. It’s
something that people don’t consider until they have 50 of them to
deal with, or 500 of them to deal with.
Walter: Exactly the issue. But I
think the savvy product developers will appreciate that and that they
are building into their systems ways to reduce the paperwork burden,
improve the workflow of the clinic along with having a new tool.
Alan: It’s worth the investment,
in my opinion. From the numbers that you’re seeing, what are some of
the expected outcomes? Let’s say, for example, using this for autism,
it’s very hard to say with a number that this was more efficient or
something. It’s almost anecdotal, but there’s got to be numbers there
that prove that this technology far outpaces anything else we’ve ever
created.
Walter: Well, part of the way we
do that in medicine is by doing long term follow up. If, for example,
we use virtual reality to help with stroke rehabilitation, both
during the acute phase after a stroke and then the post acute, when
people are coming into a clinic for follow-up or doing home
rehabilitation, if we look at how they’re doing a few years later and
compare that to people who had standard treatment, if there’s less
permanent disability, if people are returning to work sooner, if
people are able to be more functional and take care of activities —
daily living is the phrase that we use — more effectively after
they’ve used VR and AR enabled interventions as opposed to the
standard, then that tells us something and we can put a number on
that in terms of dollars saved by preventing permanent disability and
helping people get back to work sooner. Same thing for– well, let’s
use the example of treating someone who has fear of heights or fear
of flying. We can compare those people who used a VR system to master
those problems and see how they’re doing a few years later. How many
of them are still flying? How many of them need to use maybe
anti-anxiety drug in order to get on a plane, versus those who don’t?
If the ones who had the VR treatment are doing better, then that
tells us a lot and we can quantify the value of that.
Alan: Let me ask you another
question from a different angle. Have you ever seen a time where VR
wasn’t more impactful or better than the traditional way of doing
things?
Walter: That’s a good question,
Alan. We’re still in a phase where we’re mapping VR to the problems,
and I think– Nothing is really coming to mind. I’m sure there have
been situations where we thought that VR would be a better way of
approaching this problem, and it turned out to not really be cost
effective. But I have to say, I’m really at a loss to come up with a
good example. I think mostly because, again, for many years the
technology was so expensive, we were very careful to really think
through how to apply the technology the right way. Now, I should say
I’ve seen a lot of where I think it’s not necessarily always the best
use. A lot of VR technology to help people learn to relax. And some
of those VR environments I think are not that much different than
listening to an audio tape or just closing your eyes and imagining
that you’re sitting at a peaceful spot. I think that there are very
amazing mindfulness training programs that promote an active
engagement, where you’re not just at a beach listening to the waves.
And those are very exciting. But I think there are some where the
incremental value of just having a VR environment is not as amazing
as it could be, but it’s still a learning phase. We’re learning how
to leverage the technology in the best way.
Alan: One of the things I saw
was — and I think it was the Cubicle Ninjas VR meditation, it might
have been them or another one — you actually connected your
smartwatch to the VR headset so it could read your pulse. You were
able to go through these deep breathing exercises guided, it is like
a guided meditation, deep breathing and you could actually watch your
heart rate go down as you controlled your breathing.
Walter: That is a good example
of a good use case, where we’re leveraging not just the display
technology, but incorporating the measurement of psycho-physiological
signals and using it to dynamically change the environment. So like
anything, there’s a spectrum of innovation that can be applied. And I
think when you asked me where VR isn’t really that effective, all I’m
really finding is to think of our times where, it’s effective but not
as effective as it could be. If we add extra layers of analytics, or
extra layers of social connection, or extra layers of leveraging game
technology to make things more rewarding and exciting. So but again,
we’re really just getting started now. Now that the products are more
affordable, I think you’ll see a surge of people applying them to
clinical issues. And we probably will see some that are sort of
poorly designed and lame, but I’m sure we’re going to see some that
are just incredible, too.
Alan: Yeah. I would have to say
that we’ve kind of passed the point where we know which VR techniques
make people sick. So I get really, really bad motion sickness.
Walter: Uh-huh.
Alan: I’m the guinea pig for the
office. But yeah, there are still some companies out there making
things that make people sick.
Walter: Yeah, and I think that’s
a matter of poor design, if you moved the world around someone, it’s
bound to make them sick. But if you give them agency to move through
the world, it’s very different. Let me tell you about one way we’re
applaying VR and AR technology to medicine that I’m particularly
excited about. There’s a focus now in medicine to move towards more
what we call precision medicine, where instead of a one-size-fits-all
approach for treatments — where you go down a clinical pathway and
say, well, let’s try this and see if it works, and if it doesn’t
work, we’ll try something else — where we leverage the genomic
information about individual, the measurements we can get about their
behavior and their physiology. And we challenge them with virtual
environments, for example, to see how they react. And we come up with
a precision pathway for treatment that is based on their specific
biotype.
One of the projects we are in the midst
of doing at Stanford University is a project headed up by B Williams
called the Engage Project, where we’re looking at people who have
comorbid depression and weight management issues. And we’re following
them over a multi-year period of time, as they learn to manage their
weight and manage the moods. And they have a variety of different
interventions. But as they go down that treatment pathway, we’re
imaging their brains using a functional medical imaging device. And
we’re also measuring their behavior using data collected by– they
opt-in to allowing us to collect data on their smartphones about how
fast they’re typing, and how fast they’re swiping, and their general
activity levels. And then we’re also using virtual reality as a way
of challenging their neural circuits to see– we have a challenge,
much like Beat Saber, where they have to learn to react the right way
and maybe some other times control their reaction. And the idea here
is to come up with a VR based system that can be used to identify
different neuro circuit biotypes, and that can be used to come up
with a more precise approach for treating depression, for example. So
for biotype A, we might recommend that they try this treatment for
depression, and a different biotype — again, identified by how they
react to a stimulus in a virtual environment — might be recommended
to go down treatment pathway B and a third person might go down
treatment pathway C, all based on their assessment that we’re able to
do using virtual reality. So it’s a research project, but it harkens
to the vision that we can leverage not just VR technology, but also
sensor technology, biosensor technology, and machine learning
technology to really come up with a better way of adapting our
medical interventions to the specific individual.
Alan: It’s incredible. You talk
about the three pillars: training, assessment and intervention. But
one of the things that I think must be under the assessment side of
things is being able to visualize your data. As a physician, being
able to take an MRI data and blow it up and get right in there, I
think we’re only scratching the surface of that. Have you seen any
real world applications of visualizing medical data in unique ways in
VR and AR?
Walter: Oh, absolutely. For
example, there’s some groups doing virtual colonoscopies where they
collect the colonoscopy data, but sometimes it collected just using
an imaging machine as opposed to doing the actual physical
colonoscopy and then allows a clinician sort of fly through the colon
looking for polyps. Or same thing for pre-surgical planning. We can
take CAT scan data, fuse it with ultrasound data, fuse it with other
data, and then plan a complex surgical procedure in advance. So
allowing the clinicians to– not just the radiologist but also the
surgeons, to visualize in three dimensions the complex process that
they need to go through in their operation and rehearse it in
advance.
I’d like to add to those pillars that
you mentioned, too, though. It’s not just training and assessments
and interventions. I think the AR technology is also going to be very
helpful for promoting health and wellness, showing us the effect of
our behavior and promoting us to do what we really should do to be
healthy. It’s a very difficult thing to get yourself to exercise, get
yourself to eat the right way, to remember to take your medications,
especially if they have maybe side effects that you don’t like, and
to understand the consequences of not adhering to what you should be
doing to keep yourself healthy. One technique that was pioneered at
the Stanford Virtual Human Interaction Lab was to create an image of
your future self, an avatar of your future self that you could talk
to and who could talk to you, and show you in a shortened timeframe
the consequences of your decision. So if, for example, your
smartphone is sensing that you’re going into a bar and you’ve already
decided that really you want to cut back on your alcohol use, your
phone can ring, and there on your phone is your future self. Someone
who looks like you, sounds like you, but 20 years older saying, “Hey,
I thought we talked about this. Look at what you’re doing to me.”
Or maybe if you’re not exercising enough or if you’re not eating the
right way, it could show up in your future self looking very
unhealthy. Same problem for managing addictions.
So I think prevention wellness is
another pillar. I think another thing to keep in mind is I think VR
and AR technology is going to allow us to reach underserved
populations. I think the focus of care will shift from the clinic to
wherever the individual is. There’s some things, of course, we have
to do at a specialized treatment center where there’s specialized
equipment. But I think for a lot of things, especially in terms of
behavioral medicine and psychology and psychiatry, I think we’ll be
able to start doing a lot more reaching underserved populations and
be able to provide effective treatments by leveraging the
telemedicine nature of VR and AR technology.
Alan: Well, that’s wonderful.
It’s a nice close to this conversation, because if we can use these
technologies to democratize wellness and health, not just medicine,
not treating people when they’re sick, but really just keeping people
healthy, I think that is the fundamental shift that we need to make
as a society. And I think VR and AR lends a very good hand to that.
Walter: Well, and especially if
we can follow the pathways that our colleagues in the computer gaming
arena have pioneered. They know how to grab people’s attention and
keep them involved in a process. So if we can look at how they have
learned to do that in the gaming arena and apply that to some of the
interventions we’re doing in the medical arena, I think we can make
medicine not just more effective, but also more engaging and more
palatable.
Alan: And fun. Let’s not take
away the fun. I want to say, on behalf of all the listeners and
myself included. Thank you so much, Dr. Greenleaf, for taking the
time to be on this podcast. It’s been really amazing.
Walter: Well, thank you, Alan.
Thanks for all the good work you do. Getting the word out and for the
good questions today.