Keynote Address
Virtual Reality and Healthcare:
The Past, The Present, and The Future
Dr. Walter Greenleaf, Stanford University
Video | Podcast | Transcript
Transcript
Robert Fine: Well welcome everybody. Thanks very much for joining us and coming early. It’s always difficult getting downtown at the right time at 9 a.m. for a conference. But, thank you for making it out. It’s going to be gorgeous day outside today low humidity. My suggestion for you is if you can steal yourself away from the HoloLens for ten minutes, have lunch outside in the park. Lunch will be box lunches and that will be at 12:30pm break. So, first off, just really want to thank actually Dr. Pines, who started this with me and you know was very gracious to be the sponsor working with us an outside organization to bring this to GW. And I also want to thank GW and Cattleya, the event manager here. It’s a great, great space.
So just some housekeeping notes. We have the main morning session will be here in this main room. We’ll have one break at about ten thirtyish and then we’ll have a one hour lunch break in about 12:30pm. In the afternoon we have two tracks. Track one will be in this auditorium and track two will be in the downstairs auditorium. All the food and stuff will be downstairs. Unfortunately, no coffee, or food in the auditoriums. I know you have it in your hands now. Try to be very careful with it and finish it during the break. You can have bottled water.
Really, all this, we started organizing this just about two months ago and it’s really snowballed honestly in the last two weeks. And I actually am I’m very pleasantly surprised. I think more than about half of our attendees are from outside the D.C. area. We have people coming from all over. One speaker, Christopher Queens, coming from Dubai. We have an attendee that I think booked his ticket yesterday from Denmark. And we’ve got people from across the country from Arkansas to California to Texas and Louisiana State University.
On your agenda, all of the speakers Twitter names and other appropriate names are there. I highly encourage you to share on Twitter if you like. Always great to share a slide that you really like or taking a picture with your favorite speaker. The hashtag is #VRVoice. Getting on the Wi-Fi, there shouldn’t be any issues just use GW connect. Let me know if there are any problems getting onto it. And then also, we will have two raffle prizes to raffle. We’ve got four prizes actually. So, one right before lunch and one at the end of the day. You have to present when. During the break, please grab a ticket from the registration desk and you’ll be good to go.
Our keynote speaker Walter Greenleaf, who you see up here in the corner, unfortunately his mother hasn’t been doing well and he’s got to be in Erie tomorrow. He just got back from China and he’s graciously gotten up at 5:30am in the morning California time to do our keynote. And I thought it was an important enough presentation and an extremely great overview that I wanted to keep him as our keynote, even though he’s remote and can’t be with us today. So, with that I’m going to hand it over to Dr. Pines who’s going to be the emcee for this morning. I’m going to be helping him as needed technically and then we’ll get started with the keynote. Dr. Pines:
Dr. Jesse Pines: Thanks so much Bob. I’d like to welcome everyone here today to this exciting event on the use of virtual reality in healthcare. I’m Jessie Pines, the director of the Center for Healthcare Innovation and Policy Research here at GW and also a professor in the departments of Emergency Medicine and Health Policy and Management. GW chipper. Our group is dedicated to developing, evaluating and disseminating innovative practices in healthcare delivery and also in improving the value of healthcare. Chipper is housed within the School of Medicine and Health Sciences.
Just for a few words about GW and innovation. I think GW is really a very fitting place for this event. One of the first conferences on virtual reality and healthcare. Over the past several years GW has made many investments in promoting healthcare innovation through supporting programs such as chipper. Creating fantastic programs through the tech transfer office and entrepreneurship. Creating new educational programs and also promoting innovative care delivery at both GW Hospital and also GW Medical Faculty Associates, our clinical partners. And also promoting really what I think is necessary here which is the multidisciplinary collaboration that’s necessary to develop programs that are really sort of transformative in healthcare like we’re going to see today.
For today’s conference I’d like to thank Bob Fine and his team for putting together what I think will be a really fantastic program today. Today we’re going to hear about some really revolutionary use cases for the use of virtual reality and augmented reality in healthcare and trying to understand a little bit about how this new technology can improve the quality of care. Can improve patient experience and care. Potentially outcomes. And also, really improve medical education. I’d also like to thank the G.W. Milken Institute School of Public Health for hosting this conference here in this room.
I’d like to first start off with saying a few words about healthcare innovation. Innovation is one of those buzzwords that can really mean different things to different people. Most commonly we think of innovation as something like a new gadget. So obviously this classifies as something innovative. Virtual reality is a gadget. And really there’s a sort of constant desire here in the U.S. and around the world to develop new healthcare technology. And when you think about what sort of has advanced medical care over the last 20 to 30 years, it really is the new technologies that have come into healthcare. And when you think about if you’ve got something serious, some sort of life threatening illness, the U.S. is I think still today the best country to get care. And a lot of that is treatable through advanced technology that’s been developed through this sort of thirst for innovative technology. But also, at the same time we’re living in an era where the attitudes about new technology have shifted a little bit in the last 5 to 10 years. Specifically, when it comes to spiraling insurance premiums, healthcare prices and out-of-pocket costs for patients. Really, not just one of the top 10, but probably one of the top two issues facing our country today.
So how do we reconcile this desire for new gadgets and new technology with this sort of need to lower costs and really, I think to me the answer is about how understanding a technology like virtual and augmented reality can bring actual improvements in care delivery. Sort of measurable improvements in outcomes and also improve the value of care. So really, the central question I think for US healthcare with any sort of new technology or new innovation is that return on investment. And sort of the buzzword for that is called disruptive innovation which basically brings technology to bear in a way that makes healthcare more accessible and potentially cost effective. As an example, there are many disruptive innovations in healthcare. The balloon catheter created an entirely new field of interventional cardiology decades ago, making it possible to treat coronary heart disease in ways that prior to that had only really been treatable through either medications or through very invasive surgery that was costly and had a lot of complications. And soon that relatively low tech tool became more sophisticated with the addition of stents and drug eluting technology. And this was disruptive innovation in the sense that this created a new market. It was an effective procedure, and it could also be done by non-surgeons rapidly at a fraction of the cost of the more invasive procedures.
So, the question for us today is not whether virtual reality is innovative? My 9 year old son had a question: What are you doing today Dad? I said, you know, I’m going to a virtual reality conference and I showed him some pictures on the Internet and he said: You know that’s pretty cool. So, according to my 9 year old, this is innovative. So that I think there’s no there’s no question about that. But the question I think for us to answer today is: How do we really make this disruptive where we’re improving the quality of care? And I think today we’re going to hear some clues about that and how virtual reality and augmented reality can potentially be not just innovative, but a disruptive innovation, improving patient outcomes, experience and also potentially reducing costs. And also, I think we’re going to see some applications in terms of improving education in medicine. So, sit back today and enjoy to what I think may be one of the next big revolutions in healthcare delivery and education. So, with that I’m going to go ahead and introduce our speaker Dr. Walter Greenleaf from Stanford University.
Dr. Walter Greenleaf: Hi everyone, it’s very exciting for me to be here for a variety of reasons which I will actually elaborate on. I’m at a disadvantage. You can see me, but I can’t see you. So, I’m really not able to tell if you’re frowning or smiling, so please clap or hiss depending on what you think about what i’m saying. For me, the reason I’m very excited to be here is it has been a 30 year journey. I started in the field of virtual reality technology decades ago and i’ll talk a little bit more about the journey that I have had with my colleagues. As you can see the technology take off at the current pace it has and all the application areas is so very exciting.
I’ve recently been to visit some of the larger hospitals in China. I also recently got back from the University of Nebraska Medical Center where they’re spending 104 million dollars on a medical VR program. I’m spending time working with many other medical centers who are embracing the technology and for me it’s just so exciting to see it take off. But let me show you some material about what’s been going on. What is going on. And speculate a little about what will be going on in this field and then we’ll have some time for questions.
Robert Fine: So, Walter give me one second. I’m going to shrink you into the corner a little bit and then you should have mouse control now.
Dr. Walter Greenleaf: Well, when I think about the major things that we should be concerned about in our world in a global way, there’s of course political conflict, global warming, et cetera. But, the healthcare crisis in my opinion is one of the biggest looming crises that we’re facing. As you know we have on a worldwide basis an aging population and if we look at the demographics and just do the math, it’s not a matter of speculation, it’s just a matter of addition. The U.S. population and the world population has more or less doubled since 1970. However, we’re very top heavy. There’s a much higher percentage of people living longer in life and a smaller number of children to support that aging population. We just need to look 10 years down the road and it’s going to cause a major economic crisis for not only our country, but really for the world economy unless something is done.
And in my opinion, and I’m sure many of the people in the room embrace this position also, our only way out of this looming crisis is to transform healthcare with technology. The good news is there is a digital healthcare revolution going on. With all the exciting things going on with mobile health and e-health. With wearable sensors. We’re transforming medicine. And it’s moving from a process that was previously building from a clinic and clinician center to something that now, that the patient is the center of care at the center of the action. And we’re leveraging not just sensor technology, but really all the changes we’re going along with the Internet infrastructure and the emerging technology that contribute to this, we’re seeing a big transformation. And this involves AI, VR, AR. All of the two letter acronyms. Blockchain. I guess we haven’t made a two letter acronym for that one yet, but it’s a confluence of technologies that are causing this digital revolution.
Right now, every medical device is being reinvented and moved from an analog handwritten subjective environment to a more objective data driven environment. So, we’re collecting a lot of data. And of course, the quantified-self movement is the early adopters for this. People who measure everything about their life. Their sleep habits. Their exercise. Their diet. And this data when it’s been identified can be used as part of our description of prescriptive medicine. Right now, there’s a revolution in terms of apps and websites and interventions that can be driven by the digital health revolution. We have some amazingly powerful complex systems that have patient-based applications that interact with patients. Give them reminders and provide intervention. Collect better assessments.
And on the other end, we also have clinician facing dashboards. And in the middle, we have this fantastic engine that is cloud based. That is collecting data. Aggregating data and allowing us to do our analytics and real time feedback to the user and to the clinician. This provides a tremendous wealth of value. Previously, we walked along every step of the value chain. Now we’re collecting data. We’re aggregating data. And we have the tools to do better analytics. So, a major thrust in this digital health revolution are the medical applications that are virtual reality technology. My position right now is that although entertainment and gaming and social connections are going to drive the initial adoption of VR technology, we’re seeing that right now the deepest and most significant market for VR I believe is going to be in critical care.
And it will take us a few years to get there. But it’s already starting. So right now, because we’ve had more than thirty years worth of research and development of VR technology, we know in a pretty precise way what some of the useful pathways are to apply the technology, and where some of the blind alleys are. However, the cost has dropped tremendously for using this technology. We see examples of how VR has been used both in prevention and promoting health and wellness. Better assessments and evaluations. Better interpretation and better training. So, after years of research by university based clinics and teaching hospitals, the technology is finally ready to escape from the laboratory and come out into a part of clinical care. And in many ways we can look to examples of where the technology has been applied in some cases for more than 15 years. Clinically used as an intervention to understand its value.
And I’ll be specific about this. So, the overarching theme that I will have from my talk is that VR is not just a better way to train someone how to do a clinical procedure. But it is also a better way to do an assessment of a patient. A better way to promote improved adherence to protocols facilitating behavior change. We’ve had some remarkable success there in an area that’s very difficult to do. And of course, it’s a fantastic way to promote telemedicine and distributed care delivery to underserved populations.
So again, I started back in 1984 when I was a grad student at Stanford working in the Department of Physiology. Things have changed quite a bit for me since then. And fortunately, the technology has improved quite a bit too. When I first got started, the VR systems were incredibly expensive. We used a computer that was the size of a small refrigerator. And a head mount display that sometimes was so heavy, we had to counterbalance them with a brick. The technology was expensive and awkward. It was uncomfortable. But, we were able to use it for research and determine some of the useful points for the new technology. I’ve tried at several points along my pathway to see where we can bring the technology out to be useful for some immediate situations.
In 2008, I traveled to China to help after the Sichuan earthquake where they had more than 50,000 people who experienced post-traumatic stress. Unfortunately, at the time the VR technology was way too expensive to address that large population of problems. And it just was not as sophisticated and easy to use as it should have been. I believe now if we had taken the same approach, we would easily be able to use mobile based VR technology to address a large population like that. So, of course as everybody in the room is aware, now is the time for VR. Every major technology company has a focus on VR technology. Samsung, Microsoft, Facebook, Sony, Apple, Google and the list goes on. These are just a few examples.
And it’s not just the head mount display technology that the consumer electronics technology companies are making. It’s also the software system, the social interaction systems, the motion capture systems. There’s innovations and investments going on in a very significant way in all aspects of VR technology. And it’s not just the hardware manufacturers. There’s thousands and thousands of startups ranging from venture funded groups with significant investment money to two guys in a garage being very creative. So, look forward to a lot of amazing applications coming out to ride on this amazing technology. The projections of adoptions is very amazing. I think Samsung sold 70,000 systems just in the last quarter and within the next three years the projections are that VR will be adopted by 30 million users or more. This is a fantastic adoption curve. It’s faster than the internet was adopted. Faster than smartphones were adopted. So, hold onto your seats. The technology is racing toward us, and it will be an amazing ride. And by the way when I say VR technology, I really use that as a shorthand for VR and AR and mixed reality technology.
To me, it’s a spectrum of immersion that at one end, we have fully immersive systems. At the other end we have systems that capture the room and overlay it with extra information. I think because of the current state of the technology, we tend to segregate it. But, soon we’ll have systems that have an adjustable degree of immersion and we won’t be limited to two or three separate categories. And we don’t always have to wear a head mounted display to do VR. We have some room-based systems called cave systems. Although, we’re looking for a better term than cave where one or more people can experience a virtual environment and interact with the 3D environment.
So again, we’ve had a large amount of fundamental research going back over the years. Unfortunately, the premier medical VR conference which met for 22 times starting back in 1992 has recently closed itself up. The founders of that conference are retiring, but we have a history and a community of researchers and scientists who have been studying medical applications of VR for many years. And if you go to Pub Med, you can find a lot of the research. What always amazes me is that many of the new developers coming into this arena assume that VR is so new that there isn’t a history of development. But there is. And there’s a lot of really useful lessons learned that you can find from that early research. So now that the technology is less expensive, and I think more importantly it’s not just that it’s less expensive, I think there’s also been a sea change in our attitudes about VR technology.
Previously, it was something that was exotic and perhaps a little bit weird and did not lend itself to a medical arena where everybody is addressing very serious problems such as healthcare. But I think now people are viewing it as a very powerful enabling technology, and that change in attitude as much as the reduction cost is driving some of the adoption the technology. So, I’m going to pause for a second. If I was able to see you, I would pause and say: How are we doing here? Should I pick up the pace? Slow down? I’m just going to keep rolling, but please keep note of any of your questions so we can talk about it later.
I categorize the medical applications of VR into four major arenas: Health and wellness. Preventive medicine. Improved clinical training. More objective and standardized clinical assessments, particularly in behavioral medicine. And there’s a long list of medical interventions. So, let’s start with medical training. We’ve had simulators used in medical procedures, endoscopic simulators for example for a long time. But, now with the use of VR technology, we’re seeing applications being developed that are beyond just the fine details of a surgical procedure, surgical simulation. We’re starting to see it expand to the broader clinical environment. How teams can work together. How medical equipment and tools can be learned not just by going to the simulation center, but at the clinic prior to a procedure. A clinician can rehearse a procedure. Used for helping teams respond and practice responding to an emergency.
I saw yesterday a great example of a mass casualty drill where more than a thousand participants can log in. Be part of an avatar and rehearse. A police department, a fire department, a hospital emergency department and a mayor’s office of the city all working together and practicing how to respond to emergencies. And one of the unique areas of virtual reality technology is that because we can switch our point of view and see the world from somebody else’s vantage point, it’s also a very powerful tool for helping us learn skills like empathy.
So again, we’ve had for a very long time some really amazing simulators that can be used to try out procedures in a surgical environment. Though, these have been very expensive and typical scenarios for someone to fly in and spend a couple days at a simulation center to do that. It’s a new wave of technology. We can push this out and provide some degree of training. Maybe not with as much haptic feedback at this point. And one example is a system developed by OssoVR to help a clinician learn to do and rehearse using a medical device. In this case the example is placing a tibial nail prior to the procedure. So, a situation might come up where someone needs to get familiar right away with a new medical system or rehearse the procedure. This can be done with VR and I think this example is using a Vive. Let’s see if the video here works. In this case, he’s rehearsing and getting familiar with the sequencing. And more than anything else, how to use this device. There are other examples too. Here’s an example from a company called ????. They have been working in this field for 25 years now on multiuser. And this is multiuser virtual environments to rehearse and plan for a clinical procedure.
In this case, it’s an example of preparing a patient for dialysis. And you’ll notice in the corner there’s some gamification elements of their application so that one can get feedback on the right procedure. Keep score. Come up with some metrics of how a trainee is doing in terms of learning the procedure. And again, one key point is that this can be done as a team training process too. And of course, surgical training. But migrating from surgical training is preoperative planning and image guided surgery where we overlay on the surgical operating field extra information. And then you’re in a VR AI world. So, there’s a registration of the medical imaging and a clinician or a surgeon planning the procedure in advance and be prepared to go in to do a complex surgery having rehearsed the procedure and lined up all the sequences that need to be done.
Moving to the next category is better assessments. Not only can we do some measurements of physical capability much like a lab has been able to do for many years, but in a less expensive and portable manner. I think this is a really key point. To better cognitive assessments. We are moving from the day of subjective paper and pencil testing, and self-report to where we have some standardized environments that people can behave in and we can capture a rich amount of data about their function in a standardized virtual environment. Down to the point of where they are gazing and where they are specifically looking and how much time they’re doing it. This allows a whole new level of detailed assessments of cognitive function and the need for a whole new range of standards for how to interpret that data.
But, also things like active daily living assessments can be done and physical medicine will be greatly enriched by having this ability to more accurately and functionally collect data. I think that with this new range of objective assessments, we’re going to see some improvements not just in how we assess the patient, but this data will drive better pharmaceuticals and better medical devices. We’ve had a lot of noise in our data before because of the subjective nature. But with a standardized environment that can be used for very subtle assessments of cognitive function, this will drive innovations and the interventions.
So, I’ll start by being specific about some of the areas where VR environments have been used and some of these areas are new. Some of these areas we’ve had decades of research indicating what works and doesn’t work. But, it’s hard for me to actually think of an area of medicine where there hasn’t been some attempt to apply VR technology with some success. At the very least understanding what works and what doesn’t work. So, the spectrum is large ranging from stroke and traumatic brain injury and neuro rehabilitation to physical and occupational therapy and other aspects of physical medicine. But, optical rehabilitation, distraction therapy and pain management techniques using VR to address both acute and chronic pain. Facilitating behavior change. For example, weight management. The list goes on and I think the major area that I’m seeing a big surge forward is in the field of behavioral medicine. We’ve had great success at helping with addictions. Nicotine. Alcohol. Substance abuse. We’ve seen applications helping with psychosis. Great results in post-traumatic stress disorder and phobias, but also advances in mood disorders. Depression. Anxiety disorders.
We’ve had some modest success at looking at mild cognitive impairment and age-related cognitive decline. Applications in the field of autism and also attention deficit disorder. But the list goes on. Phobias. Obsessive compulsive disorder. Anger management. Conduct disorders. Compulsive disorders. Learning disabilities. It is just amazing the spectrum of applications and again in particular in the field of behavioral health medicine which is really ripe for innovation.
Switching from better assessments and interventions, it’s also likely to see the applications of VR to preventive medicine and promoting health and wellness. There are systems out there being developed for helping with weight management. Promoting people to exercise both as an individual or as part of a group. I saw a fantastic system yesterday which guided people to do a tai chi system and not just as an individual, but you could be doing as part of a group. Amazing systems for stress management. Interfacing with people with a disability. And we’re seeing some great strides forward in addressing senior isolation providing grief counseling and palliative care and also helping with anxiety and depression.
And I think it’s worth pointing out that although the best use of the technology now in terms of voracity of the environment and a feeling of complete immersion, VR is also a mobile platform. This allows us to reach outlying areas that otherwise we would not be able to easily reach. And sometimes remote areas, not just places physically remote, but an area that’s difficult for patients to make it into see a doctor. Some cities have a problem with access to underserved populations. VR technology when implemented as part of a mobile telemedicine platform can transcend some of these barriers. And in a less expensive way.
As I mentioned earlier, it’s not just the VR technology, the head mount display, and motion capture technology. Our field is being greatly enhanced by the use of AI technology. This is an example of a project at USC’s office of technologies. I’ll show a video clip here. I want to describe what’s going on before I show it to you. In this case we have an AI system represented by an avatar of a woman who is talking with a patient. She is having a dynamic interview with a patient and showing a way to interview a patient for behavioral medicine. But it’s being complemented by a capture of information biometrics from the patient. So, gaze direction. Tone of voice. Content of voice. Body position. Often non-verbal clues about the patient are also being analyzed dynamically. So, this is a great example of how an AI system can compliment a VR system to use for medical training.
And I should also point out this is something of a situation that could be flipped around where instead of having a situation where the AI system is interviewing the patient, where a physician could practice and learn how to interview a patient. Perhaps a difficult patient. Practicing on a virtual patient. So let’s take a look at the video:
Ellie: Hi, I’m Ellie. Thanks for coming in today. I was created to talk to people in a safe and secure environment. I’m not a therapist. I’m here to learn about people and would love to learn about you. I’ll ask a few questions to get us started. And please feel free to tell me anything. Your answers are totally confidential. Are you okay with this?
Patient: Yes.
Ellie: So, how are you doing today?
Patient: I’m doing well.
Ellie: That’s good. Where are you from originally?
Patient: I’m from Los Angeles.
Ellie: Oh! I’m from L.A. myself. When was the last time you were really happy?
Patient: When was the last time? I don’t know. I’m not someone who’s really like…I don’t have any real high highs. I feel like a…
Dr. Walter Greenleaf: So, you get the idea there that this is a fantastic system that can be used for clinical training and maybe one day systems like this can also use for try outs. Where someone can call up through a cellphone or through a website, interact with a virtual agent, get some advice and get some information. As these avatars become more believable and as we get better at natural language speech recognition, there’s tremendous potential to compliment face to face interaction with AI. So, let me jump in by giving you some specific examples of where VR technology is currently being used. One of the hallmark areas that we’ve been using VR for is post-traumatic stress disorder, phobias and other anxiety disorders. This goes back to the early 90s with a virtual Vietnam that we had. Now, Skip Rizzo’s group has developed, and we’ve deployed, and I think up to about 60 VA hospitals, a virtual Iraq, a virtual Afghanistan. We also have used VR very successfully to help with fear of flying. Fear of heights. Fear of spiders. Social anxiety disorders all have been implemented in some very robust VR environments that can be used for gradual exposure therapy to take the patient one step at a time in a reproducible way to address their traumas and their fears.
And again, along the way we can actually collect a lot of data about utilization, and that data can drive further improvements to the intervention. It’s an amazing way to take the standard imaginative exposure therapy where people are asked to imagine their worst fear and their worst trauma and transform it into something where the clinician has control of what the patient is thinking cognitively. Something you can’t do with standard imaginative therapy. We can also use VR to drive some data driven assessments. The next video clip we’ll see will show a system that’s being used as a more ecologically correct functional evaluation system for attention deficit disorder. It’s a virtual classroom with a number of distractions in it and a standardized cognitive test. We’re collecting data from the patient. Well, in this case the user’s point of view and putting into a data analytics system that will only get more powerful as we collect more and more examples from patients.
Narrator: The virtual classroom attention evaluation system consists of a virtual classroom environment and utilizes a standard continuous performance test with multi-modal instructions. As well as motion detection of the head, arm and leg. The user takes a 13 minute test in the VR environment and the test results as well as all movement data are recorded. A test report was issued immediately that contains a personalized data analysis with clear data individualization. This may help parents receive a fast screening and assist clinicians to evaluate the ADHD diagnosis effectively.
Dr. Walter Greenleaf: I think the key phrase there it is objective and reproducible assessments. I mentioned at the beginning of my talk one of my concerns about the age related health crisis. And nicely enough, a lot of the applications of VR technology do address the problems inherent in senior care. We are not only able to do improved assessments and diagnostics, but we can use the technology to help address senior isolation and loneliness. Acute and chronic pain. We’ve had some great results there. Depression and anxiety disorders. Stroke rehabilitation, physical therapy, designing for disabilities. Staff training. VR technology lends itself very nicely to the problems that we have in senior care.
Another example of a good use case of VR technology is helping with substance abuse and addictions. In this case, the VR environments of abuse are used very successfully to teach refusal skills and situational confidence. We have virtual bars. We have virtual party scenes. We have a virtual crack house and the clinician can work with the patient to have them learn situational confidence and practice saying no to peer pressure for example. You can evoke the cravings and that cognitive state, a clinician is not in a very good position to teach how to manage those cravings. It’s just an imaginative experience otherwise. With a virtual environment, we can reproducibly and significantly evoke the cravings and help the patient to manage those cravings in a situational context.
Another example is how VR has been used for what we call stress inoculation. Preparing first responders how to deal with the traumas that they’re invariably going to be seeing. One project I’m particularly excited about is one that we are in the process of collecting data on right now at Stanford Children’s Hospital. This is a project that we call project Braveheart. We’re working with children who are scheduled for a cardiac procedure. We know weeks in advance what the worst day of their life is going to be and we can help prepare them and their family members for that day by using VR environments. I’ll show you a short video clip. But, essentially what we’ve done is we’ve captured 360 video of the whole patient experience starting in the parking lot. We’ve green screened along the way guides. Other children who’ve had the same procedure who can show them what’s going to happen at each room. Talk to them about what the experience will be and we bundle in with that some VR based relaxation skill systems so that they can practice diaphragmatic breathing. We’ve gamified that a little bit so they can learn mindfulness and other relaxation techniques, and we give it to them weeks before the procedure to take home and get familiar with at home. We’re collecting data along the way. Our study includes quite a large number of both subjective and objective measurements of anxiety. We haven’t finished the study yet but from what I can tell just from hearing what patients say, it’s tremendously beneficial for them to reduce their situation anxiety by knowing what they’re going to expect on the day of surgery. Even just knowing the layout of the hospital is helpful.
Narrator: Braveheart is an interactive VR experience developed to reduce stress levels in children who will be undergoing a surgical procedure. Patients and their families are guided through a 360 degree video tour of the hospital by a virtual companion. The gender specific tour guide provides a friendly, approachable and relaxing tour of the facility and introductions to doctors and key hospital staff. Along the hospital tour, patients visit the key rooms they’ll encounter on the day of their procedure to help make the unknown more familiar. Children learn about parts of the procedure and can even practice mindfulness and relaxation exercises.
The intuitive software design allows the user to navigate through scenes and applications learning the hospital layout, all within Braveheart at their own pace, creating a first of its kind interactive and immersive video environment. When Braveheart is finished, anonymous usage analytics are stored and available by scene, time and engagement for analysis by doctors and researchers.
Dr. Walter Greenleaf: The data collection is a strong part of this. We’re not just providing a stressful inoculation procedure, but we’re collecting data about what is the most anxiety provoking part of the patient experience. I’m going to pick up the pace here a little bit and continue with some of the examples of how VR has been used very effectively as a method of pain distraction. It’s been shown that in some cases, it reduces the use of opiate narcotics in burn treatment scenarios significantly. We’ve done neuroimaging studies to understand what is the aspect of how VR is working and what changes. And using that feedback helps us refine our interventions. The other way we’ve used VR is to help clinicians practice and get ready for how to give bad news to people. Here’s a quick video clip of a system we’ve developed to help physicians practice delivering news about a terminal illness. In this case, instead of a using computer generated avatar, we used a video avatar so that we can get the micro-expressions of the face.
Patient: I’m so scared of getting breast cancer. I don’t know how I can handle this. It’s just really hard news to hear.
Dr. Walter Greenleaf: It’s an example of how one can use the system for not just training, but also to teach empathy and help a clinician learn what it’s like from the patient’s perspective. We can reverse that process, and have the clinician be the physician’s patient. There’s a lot of exciting investments going on in VR health startups. Limbix is a Sequoia venture capital backed company that is dealing with exposure therapy. Reflexion Health and Mind Maze are both going into using VR for neural rehabilitation. Remarkably enough, Mind Maze raised 100 million dollars at a one billion dollar valuation last year. So, the investments into this arena are significant. A company that I work with is using VR as part of a combination therapy, so we find pharmaceutical interventions with VR interventions to get synergistic efficacy. And there’s a whole neuroscience behind how VR promotes behavioral change.
For the sake of moving forward fast so we have time for questions, I’ll just skip over that pretty rapidly. Essentially, we’re leveraging many aspects of the brain’s reward system with increased attention. We started the feedback loop by showing progress of the patient and we leveraged motor neuron systems to promote behavior change. One way we do that for example is to show the consequences of one’s behavior by age progressing or changing a personal avatar. We have a procedure where we have you bind with your avatar. There’s an effect called the crotius effect that we can use to have people feel that the avatar represents who they are and then we can show the person dynamically and shorten the feedback loop so they see the consequences of their actions. The idea here is to have a healthier and happier future self. And least you think this is just an idealistic concept, we did a study with some undergraduate students and believe me it’s very scary to show a Stanford undergraduate student what they could like in 10 or 20 years.
I’m very motivated for them to change their behavior. But we actually gave them real money and even a choice of putting aside for retirement. Those students who bonded with their future self, put a significant amount of money aside for their retirement account. And this has been reproduced in weight management and exercise and also with changing substance abuse behaviors.
So again, to summarize, VR technology is not just something that’s jumped out of the clamshell in the last five years. We have decades worth of research supporting its utilization in the medical arena. It’s now becoming affordable. It’s a technology that’s racing toward us really fast and will soon be part of how we work and how we interact with other people on a daily basis. This gives us a fantastic opportunity to shepherd this emerging technology and make it useful in medicine. I think we can expect that VR technology will be even better in near future and integrated into our healthcare system.
I’d like to invite everybody to come to Stanford. It’s hosting a conference on innovations in psychiatry and behavioral medicine in October with a specific focus on virtual reality technology. I’d love to have people attending this conference be part of this conference either as sitting in the audience or perhaps participating as speakers who are exhibiting some of their technology. Okay, that’s my talk. I don’t have a good sense of the room. You’ve been watching me. I don’t see you, but I’m pleased i’ve had a chance to present to you.
Dr. Jesse Pines: Thanks so much Walter. So, for folks who want to ask questions, please come up to the microphone so Walter can hear you.
Dr. Haroon: Good morning and thank you for giving us such a hope with that presentation. I’m Dr. Haroon. Everybody calls me Doctor H. I’m actually a USC graduate and faculty and I’m a child and adolescent psychiatrist. And I was getting very hopeless treating all the conduct disorders in autistic kids which we have not actually done a great job with pharmaceutical treatment. So I wanted to ask you all the kids you have either done research. Are they going to be complementing treatment with pharmaceutical treatment or we will just do VR? Or will be doing collaborative treatment with pharmaceutical treatment or pharmacological treatment and VR at the same time? When we are doing the most difficult and treatment resistant patients we have in our population, i’m in Hot Springs, Arkansas. So, that’s a very tough population to treat all the pharmacology resistant treatments with autism and conduct disorder. Those two populations. When we will be doing VR, would we keep them on treatment? Pharmacological treatment and VR together in parallel? Or we will just take them off? We feed them all from pharmacology and just do? Similarly with TBI. I’m doing some volunteer work with the veteran population. Same question goes for PTSD patients and TBI patients also. Would we just do VR?
Dr. Walter Greenleaf: The answer is that VR will be used in combination with standard procedures. We’ve seen that with treatment in post-traumatic stress and autism and Asperger’s and traumatic brain injury, it could facilitate the process, but it does not displace. With autism and Asperger’s, we can use VR to teach people. We can exaggerate facial expressions and tone of voice and how people practice confidence in a complex environment. They can learn those skills in a VR environment. Translate them out into the world. But it does not displace all the other interventions were using. It complements that. Labeling for traumatic brain injuries. We can facilitate the rehabilitation process both cognitively and motor rehab by breaking rehab down into small steps. Providing more quantitative feedback and motivating patients by making it a game or making it a social endeavor. But it has to be driven by the clinician. It’s something that snaps on top of the existing clinical protocols but doesn’t displace them.
Lauren O’Neil: Hi Dr. Greenleaf. Thank you for your presentation. My name’s Lauren O’Neil. I have a question about the PTSD TBI treatment. You mention ten years ago being in China and fielding some trauma VR content. I’m really interested in, you know, we have a certain perspective toward treating PTSD, TBI, exposure therapy and it’s with a limited number of senses and with VR, you know, you have sort of a wealth of data coming from a wealth of different, you know gloves. You’ve got olfactory, you’ve got all kinds of sensory inputs and I’m wondering how fast the feedback is moving from experimentation in high, sort of multiple sensory environments to the content producers and where those people are working the fastest with those feedback loops with actual therapists. So, where the confluence is having? Because I feel like, you know, with it moving from two to six different sensory inputs, we’re going to find things in the correlations over time especially with streaming inputs and streaming data that we haven’t even anticipated when it comes to impacts on the limbic brain. And so, where is that happening? What have you seen in the past 10 years that really kind of blew your mind? I think that’s my question essentially.
Dr. Walter Greenleaf: Thank you so much. You’re absolutely right. With VR we can have a multi-modal intervention such as having people imagine things. We can provide olfactory clues. We can provide auditory cues. And it’s a very powerful technique for exposure therapy and motor rehabilitation. In terms of your question of what do I see going on with the startups. I went to a Palo Alto based startup just two days ago called Limbix. And they’re ex-Google, ex-Oculus. Very smart engineers and they’re leveraging the huge amount of content we have from Google Maps with their 360 video to create a fantastic system for helping with trauma and driving. And it’s a very rich environment and data driven. So, I think not only will we have more modalities, we’re going to have a lot more in terms of data analytics to come up with more personalized medicine approaches. I wish I could tell you how soon it’s going to be adopted, because that’s a very hard thing. For many years, i’ve been saying five years from now, five years from now. But the systems are being developed with very aggressively by some of these startups. There was recently an FDA approval of a VR-based rehabilitation system. So, it’s coming out pretty fast.
Pat Sloane: Hi, I’m Pat Sloane from Hampton University School of Nursing. I was very favorably impressed at the Library of Medicine. The kind of things they were doing at that point back in ’74 with these huge platters. Looked like 78s that they would have that we would be looking at what was going on. I guess it was a precursor to what you were describing with the Vietnam reality. I’m a public health nurse. I’m curious as to are you doing anything to partner with perhaps others at Stanford, others throughout California, who are making the kinds of things that we saw with the avatar? So that it gets out to the public quickly in terms of…they see it as unreal TV. But, it shows effectively how these things can work to speed the adoption by the public when you are able to present it clinically. Thank you.
Dr. Walter Greenleaf: Thank you Pat. That’s a great question. I think actually nurse’s training is one area where we’ve had some fantastic results using VR as part of a training system all along the care continuum. In terms of getting it out to the public…in a way I think we have a challenge here because it’s easy to deploy a medical VR system. I think that our challenge is going to be there’s going to be a lot of junk out there. Just like people can download a lot of apps onto their phone which may or not have been validated as a proper intervention. I think the barrier between developer and user is becoming more and more frictionless and rapid. And that worries me in terms of validation and all quality control for medical interventions. So, I guess my answer to your question is yes, there’s a lot going on which is out to the public as a preventative medicine easily accessible system. That’s also a problem.
Jeff Bail: Dr. Greenleaf, I’m Jeff Bail with BostonVR. I’d like to know how medical VR can help people in the third world?
Dr. Walter Greenleaf: That’s a good question. Well, in many ways the underdeveloped countries are well positioned to leverage this technology because they have already started to adopt a telemedicine paradigm. So, I don’t think they have the same barriers and such an acute need. And I’ve had inquiries from a large number of countries saying: How can we use this technology? Because we can use a system like Google cardboard with our cellphones, how can we get this technology out as a telemedicine platform? So, it is of strong interest to countries that are trying to redevelop their medical infrastructure and leverage this technology as a better point of care solution.
Dr. Neil Martin: Wonderful talk professor. I’m Neil Martin, i’m a neurosurgeon here in Pennsylvania and you know the number one problem that we’re facing in healthcare is chronic disease. And a shocking amount of that is, you know, to put it perhaps unkindly, self-inflicted. It’s lifestyle issues. It’s obesity, substance abuse, lack of exercise. I mean, those three right there could probably you know work better than any medication and can probably cut some of our three plus trillion dollar a year budget for health care radically. How do you see this moving into a broad enough swath of the population? So that we can…because the behavioral changes required to mitigate those things are very difficult for doctors to instill. How do you see that evolving with the aid of VR?
Dr. Walter Greenleaf: Well, the good news and I hope people in the audience have had a chance to try a high-end VR system. The capabilities of the higher systems are migrating down to mobile VR rapidly as we come up with faster and faster processors in our cellphones. But there can be a very profound effect to motivate people to change. And we’ve seen some really amazing results of applying VR for behavior change in some of the areas you mentioned such as addiction and weight management. In our lab we have a little demo where people walk a plank over a height and I’d say maybe a third of people have trouble walking across that virtual plank, even though they’re in a room with other people and talking to those other people, they know it’s not real. It’s just our visual system and we have enough clues to become so believable that it activates that fear response. And once you activate a common response, you can teach people how to change the reaction. With feedback, help them learn to modulate their response. So, we’ve got a powerful tool. I’m very optimistic that we can apply to some of the problems you mentioned. But, you know, you mentioned you’re a neurosurgeon and one idea I want to run by you is for awake neurosurgery. It’s a very scary procedure for the patients of course. It would be interesting to me to see if we could create a VR system where better assessment can be done while the patient’s awake during neurosurgery. But, their mom or their brother or their best friend could be in the waiting room yet be with them virtually in the operating room. So, it’s just an idea of how we might apply VR technology.
Jenny Hughes: Hi Dr. Greenleaf. My name is Jenny Hughes. I’m a psychologist at LSU Health Sciences Center and the trauma psychologists are a level one trauma center in New Orleans. Our team is working to develop a virtual reality exposure therapy for gunshot victims in New Orleans. We have a lot of crime and violent crime in New Orleans so it’s a real need in our community. One concern that our team has is the realism of the environment and what kind of impact that might have or not for our participants. So, I’m wondering if you have any recommendations? If that’s ever been a concern for you or other groups that you’ve worked with? Or other ways like you were saying in the last question to activate that fear response so that I can do my job to help them deal with the anxiety wall and experiencing the exposure.
Dr. Walter Greenleaf: That’s a great question. There was a study done by Georgina Cardenas of Mexico City looking at urban crime and PTSD treatment there. Much of the work on PTSD has been on combat PTSD. That’s where a lot of the funding has come from. But she looked specifically at urban crime. What’s nice about virtual reality is that we can have a graduated exposure therapy. I’m asking you to use your imagination to think about the gunshot experience that you have. It’s sort of binary. You’re either not going to be shocked to think about it, or you’re going to be very upset by it. With graduated exposure therapy, we can in a controlled manner expose a patient to the scenario that the clinician wants them to see. An example for fear of flying exposure therapy is that we start by having the patient look at the counter where the tickets are on the counter or think about going to the airport. Then we have the taxi, and then we have the check-in line and then we have waiting to get on the plane and we can do it gradually with feedback from patients both physiologically and verbally and have a graduated exposure experience. That’s a very exciting application. I’m so glad you’re doing it. There are some groups I can introduce you to that might have some enabling technology. Let me know.
Jonathan Shipman: My name is Jonathan Shipman. I’m not a doctor. I come from a VR studio. I have two different questions. One was in watching a lot of the presentation, obviously a lot of the animation and the avatars are relatively rudimentary. How important do you think it will become for those beings and the things that we’re representing to become more realistic? More photo realistic?
Dr. Walter Greenleaf: That’s a very good question. You’re absolutely right. A lot of my clips are from things I’ve been using for a long time. So, some of them are very old. But you know what? They work. And I think they’ll work better as we develop better graphics and VR studios like yours are going to be an important part of that. I also think that what’s acceptable to the user is going to migrate as the VR environments get better. Right now, I think someone in a VR environment that maybe it’s five years old, but it still evokes a cognitive response because it’s new to them. The emotional valence is very high. But once we were using VR a lot for work, for social interaction, our criteria of what is realistic I think is going to attract that. We’re going to have more and more robust environments, especially with social VR. And that’s one of the big problems we have right now is multiuser search environments that have good facial expressions and nonverbal pagers. That’s where we need your help.
Jonathan Shipman: Yeah, great we’re happy to do it. Just a technical question which came up. We’ve worked with a couple of other companies who brought up the question around the length of time that a person can wear a headset and how that might affect their cognitive thinking. You know anything about that?
Dr. Walter Greenleaf: It’s an interesting issue and it brings up a related issue too. Specific to your question. You know, I think it’s uncomfortable right now with current technology. Especially mobile technology to spend very much time in a VR environment. I really do miss the real world. But it can be a little disorientating. We’re getting better at that. But one unexplored issue is what about the developing brain? What about the children whose parents have two jobs, and they’re going to park them in front of a VR system for extended hours of the day? What about the vestibular ocular in the developing brain? We don’t have the answer to that. Now, i’m personally optimistic that we can develop systems that are not unhealthy, but it is of concern. You’ll see in some of the systems they recommend not for use in people under 13. I think that comes from the fact that Facebook has that as a threshold for age. It’s not based on anything we know medically from research, but we also know that children are not good at separating what a real experience is from imaginable. So, there are some issues we have to address here.
Steven Guzman: Hi Dr. Greenleaf. My name is Steven Guzman, VR developer at Oral Roberts University in Tulsa, Oklahoma. We have a lot of faculty members that are looking for a therapy-like cave experience. We’re sitting down with a patient and they begin to react randomly. We’re looking to set up our students to go into the workforce and to be ready within that space. So, within your experience working with virtual reality, have you seen anything like that or are you guys developing things like?
Dr. Walter Greenleaf: Some of my colleagues are developing some amazing cave environments. I wish I’d included a few pictures of them. And I hate the term cave. I think we have to come up with a better term. Especially I’d like to see a room-based VR environment in every senior living center so that we can do exercise and rehabilitation and connect people. But it’s not going to work if you call it a cave. But to your point. Yes, there are groups developing some really interesting systems for training people in all aspects of work. I think it’s an important area. I think getting beyond the HMD is really key. So, I’d be happy if you drop me an email, I’ll be happy to introduce you some people that I do know that working on this.
Dr. Jesse Pines: Thanks so much Dr. Greenleaf for a really fantastic presentation.
