On Tech & Vision With Dr. Cal Roberts

This podcast is about big ideas on how technology is making life better for people with vision loss.

The white cane and guide dogs are long-established foundational tools used by people with vision impairment to navigate. Although it would be difficult to replace the 35,000 years of bonding between humans and dogs, researchers are working on robotic technologies that can replicate many of the same functions of a guide dog.

One such project, called LYSA, is being developed by Vix Labs in Brazil. LYSA sits on two wheels and is pushed by the user. It's capable of identifying obstacles and guiding users to saved destinations. And while hurdles such as outdoor navigation remain, LYSA could someday be a promising alternative for people who either don't have access to guide dogs or aren't interested in having one.

In a similar vein, Dr. Cang Ye and his team at Virginia Commonwealth University are developing a robotic white cane that augments the familiar white cane experience for people with vision loss. Like the LYSA, the robotic white cane has a sophisticated computer learning system that allows it to identify obstacles and help the user navigate around them, using a roller tip at its base. Although it faces obstacles as well, the robotic guide cane is another incredible example of how robotics can help improve the lives of people who are blind or visually impaired.

It may be a while until these technologies are widely available, and guide dogs and traditional canes will always be extremely useful for people who are blind or visually impaired. But with how fast innovations in robotics are happening, it may not be long until viable robotic alternatives are available.

The Big Takeaways:

• Reliability of Biological Guide Dogs: Although guide dogs have only been around for a little over a century, humans and dogs have a relationship dating over 35,000 years. Thomas Panek, the President and CEO of Guiding Eyes for the Blind, points out that there will never be a true replacement for this timeless bond. That being said, he thinks there is a role for robotics to coexist alongside biological guide dogs, and even help augment their abilities.
• LYSA the Robotic Guide Dog: LYSA may look more like a rolling suitcase than a dog, but its developers at Brazil's Vix Systems are working on giving it many of the same functions as its biological counterpart. LYSA can identify obstacles and guide its user around them. And for indoor environments that are fully mapped out, it can bring the user to pre-selected destinations as well.
• The Robotic White Cane: Dr. Cang Ye and his team at Virginia Commonwealth University are developing a Robotic White Cane that can provide more specific guidance than the traditional version. With a sophisticated camera combined with LiDAR technology, it can help its user navigate the world with increased confidence.
• Challenges of Outdoor Navigation: Both LYSA and the Robotic White Cane are currently better suited for indoor navigation. A major reason for that is the unpredictability of an outdoor environment along with more fast-moving objects, such as cars on the road. Researchers are working hard on overcoming this hurdle, but it still poses a major challenge.
• The Speed of Innovation: When Dr. Ye began developing the Robotic White Cane a decade ago, the camera his team used cost $500,000 and had image issues. Now, their technology can be run on a smartphone – making the technology much more affordable, and hopefully one day, more accessible if it becomes available to the public.

Tweetables:

• "We've had a relationship with dogs for 35,000 years. And a relationship with robots for maybe, you know, 50 years. So the ability of a robot to take over that task is a way off. But technology is moving quickly." — Thomas Panek, President and CEO of Guiding Eyes for the Blind
• "Outdoor navigation is a whole new world because if you go on the streets, it could be dangerous. You have to be very careful because you are driving a person, driving a human being." — Kaio Ribeiro, Researcher at Vix Systems
• "The first … camera we used, it's close to 500 grand. … But now … the iPhone's LiDAR … works outdoors. … And it just took … a … bit more than 10 years." — Dr. Cang Ye, Prof. of Comp. Sci. at Virginia Commonwealth University and Program Director, National Science Foundation
• "It's not the traditional … robot … that's stiff. … We have to move into soft robotics … to accomplish the … activity … a dog can accomplish. … It's a way off. … If … an engineering student … wants to get into soft robotics, … that's where it will be." — Thomas Panek

Pertinent Links:

• Lighthouse Guild
• Guiding Eyes for the Blind
• LYSA Robot Guide
• Robotic White Cane

This podcast is about big ideas on how technology is making life better for people with vision loss.

Navigating the world can be difficult for anyone, whether or not they have vision loss. Tasks like driving safely through a city, navigating a busy airport, or finding the right bus stop all provide unique challenges. Thankfully, advances in technology are giving people more freedom of movement than ever before, allowing them to get where they want, when they want, safely.

Smart Cities are putting data collection to work in a healthy way by providing information to make busy intersections more secure, sidewalks more accessible, and navigation more accurate. They're providing assistance for all aspects of travel, from the front door to the so-called "last hundred feet," while using automated technology to make life easier every step of the way.

And although fully autonomous vehicles are still on the horizon, the technology being used to develop them is being applied to improve other aspects of life in incredible ways. These applications are making the world more accessible, safer, and better for everyone, including people who are blind or visually impaired.

One example of this is Dan Parker, the "World's Fastest Blind Man," who has developed sophisticated guidance systems for his racing vehicles, as well as a semi-autonomous bicycle that could give people with vision loss a new way to navigate the world safely and independently.

The Big Takeaways:

• Smart Cities. Greg McGuire and his team at MCity in Ann Arbor, Michigan are working on the concept of Smart Cities, which focus on using data to improve the everyday lives of their citizens. That means improving traffic intersection safety, greater accessibility options, providing detailed "last hundred feet" guidance, and much more.
• Autonomous Driving. In a perfect world, self-driving cars will provide ease of transportation for everyone, and create safer, less congested roads. That technology isn't there yet – but it's being worked on by talented researchers like John Dolan, the Principal Systems Scientist at Carnegie Mellon's Autonomous Driving Vehicle Research Center. Sophisticated sensors and advanced robot-human interfaces are being developed to make self-driving cars possible.

• Application of Technology. Even though the technologies behind Smart Cities and autonomous vehicles are still being developed, they can still be applied to everyday life in exciting ways. Things like miniature delivery robots that can deliver goods, AI-powered suitcases that can help you navigate busy airports, or semi-autonomous bicycles are already here – and there's more on the way.
• The World's Fastest Blind Man. When professional race car driver Dan Parker lost his vision in an accident, he felt lost. But a moment of inspiration led him and his business partner Patrick Johnson to develop a sophisticated guidance system that let him continue racing without human assistance. Thanks to this revolutionary technology, Dan became the "World's Fastest Blind Man" when he set a land-speed record of 211.043 miles an hour in his customized Corvette.

Tweetables:

• "One of the key pillars of MCity is accessibility. The four areas we think about are safety, efficiency, equity, and accessibility. … Accessibility is that we can make transportation systems available to as many of us as possible." – Greg McGuire, Managing Director of MCity
• "I became the first blind man to race Bonneville, with an average speed of 55.331 mph. And I returned in 2014 and set my official FIM class record … at 62.05 mph. … I'm the only blind land speed racer … with no human assistance."– Dan Parker, the "World's Fastest Blind Man"
• "There are chairs, there are tables. ... We know we don't want to run into them, but we do want to walk in the walkable space. … A car wants to drive in the drivable space." – John Dolan, Principal Systems Scientist at Carnegie Mellon's Autonomous Driving Vehicle Research Center
• "Because we know autonomous technology is increasing every day and it's coming, you know, a hundred percent it's coming. You know, transportation is freedom and that's exactly what that would bring us. Freedom." – Dan Parker

Contact Us:

Contact us at [email protected] with your innovative new technology ideas for people with vision loss.

Pertinent Links:

• Lighthouse Guild
• MCity
• Carnegie Mellon Autonomous Driving Vehicle Research Center
• Dan Parker

This podcast is about big ideas on how technology is making life better for people with vision loss.

For decades, people with vision loss had limited options when it came to accessing video games. Aside from screen magnification and text-to-voice tools, gamers who are blind or visually impaired didn't have many ways to play their favorite titles. But in recent years, the same cutting-edge technology used to create games has been used to also make them more accessible for people with vision impairment. These advances include more visibility options, the implementation of 3D audio, haptic feedback, and customizable controllers for gamers with vision impairment. Furthermore, 3D audio technologies being developed in live sports may soon make their way to online multiplayer video games. The implementation and improvement of these technologies mean that everyone will be able to play together, regardless of their visual acuity.

The Big Takeaways:

• Leap in Accessible Gaming Options. In the past, the lack of accessibility features has made it much harder for gamers like Elaine Abdalla to access her favorite titles. But as gaming technology and accessibility improve, gamers like Elaine who are visually impaired are now able to more actively participate in their favorite titles. For example, The Last of Us: Part 2 boasts over 60 accessibility options including sophisticated visibility adjustments and inclusive difficulty settings.
• Participating in the Process. Robin Spinks is the Head of Inclusive Design at the UK's Royal National Institute of Blind People (RNIB). He helps gaming companies implement accessibility options as part of the design process from the very beginning. By ensuring people with vision loss are part of the process from the jump, Robin is helping make games more immersive than ever — for everyone.
• Xbox Accessibility Team. Kaitlyn Jones first became passionate about gaming when she helped her father launch a nonprofit foundation that built customized setups for gamers with disabilities. She's now the program manager of Xbox's Gaming Accessibility Team. They develop guidelines for all kinds of accessible gaming options, including for people with vision loss. Thanks to her team's efforts, new titles include options like spatial audio cues, which make it easier for gamers who are blind to navigate complex dungeons and unlock achievements.
• Action Audio. At the 2022 Australian Open, a new technology called Action Audio became available for tennis fans with vision loss. Through cutting-edge camera tracking technology, Action Audio creates an information-rich 3D audio experience that allows tennis fans with vision loss to experience every thrilling rally as it happened, and to share in a communal experience.
• Spatial Audio in Gaming. Action Audio's developers hope to make it a universal technology that can also be used in gaming. Whether it's working with a team in the latest first-person shooter or dribbling around opponents on a virtual basketball court, 3D (or spatial) audio technology is positioned to help people with vision loss to more equitably participate in the gaming community.

Tweetables:

• "The motivation here is to remove barriers, to form partnerships, and to collaborate so that going forward it becomes a much more possible area of life for people. 'Cause gaming's fantastic, right? And why should you be stopped from enjoying it just because you're blind or partially sighted?" — Robin Spinks, Head of Inclusive Design at RNIB
• "When something happens on the court, it's captured quite fast with Hawkeye within hundreds of milliseconds, and we're able to take that data relatively quickly and generate the audio and then send it to the broadcast before it's received in the real world." — Tim Devine, AKQA Executive Director of Innovation
• "It's like that Japanese principle of continuous improvement, kaizen, where you're kind of constantly looking to do things better. That's what we want to see in the gaming world. — Robin Spinks, Head of Inclusive Design at RNIB
• "We've always really prioritized accessibility along the way. But I think in terms of the journey of, even where we started when I first joined the team a few years ago versus now, I think the bar honestly just keeps getting higher and higher." — Kaitlyn Jones, Program Manager: Xbox Accessibility Team
• "I'm just happy to see the disabled community is finally getting the assistance they need. It's taken a while and it's still gonna take a while, but we're going in the right direction." — Elaine Abdalla, creator of BlindGamerChick YouTube Channel

Contact Us:

• Contact us at [email protected] with your innovative new technology ideas for people with vision loss.

Pertinent Links:

• Lighthouse Guild
• RNIB Accessible Video Games Page
• Xbox Accessibility Guidelines
• BlindGamerChick YouTube Channel
• On Tech & Vision: Training the Brain: Sensory Substitution

This podcast is about big ideas on how technology is making life better for people with vision loss.

Marcus Roberts, Stevie Wonder, Ray Charles, and even Louis Braille (who invented the Braille Music Notation system still used today) prove that musicians who are blind or visually impaired have made profound impacts on our musical landscape. However, to get their work to us, musicians who are blind have had to structure complex workarounds, like relying on sighted musicians to demonstrate complex scores; memorizing long pieces; or only performing when they can have a Braille score in front of them, shutting them out from opportunities that fall to those who can sight read, since Braille scores have often been time-consuming and expensive to produce. However, new technologies in music composition and production are making composition, nuanced scoring, and Braille printing easier than ever, bringing musicians and composers who are blind to centerstage to share their sound and song.

The Big Takeaways:

• "Lullay and Lament" by James Risdon. The recorder — pivotal in music from the Renaissance and Baroque periods — only lately has emerged from a long period of obsolescence. James Risdon, a passionate player who lost his vision due to Leiber's congenital amaurosis as a child, has written the original song for recorder "Lullay and Lament" for his album Echoes of Arcadia, which marries the early recorder with contemporary recorder music. To make this album, he relied on new musical technologies like Dancing Dots.
• Dancing Dots with Bill McCann. Bill McCann is the founder and president of Dancing Dots Braille Music Notation software. Dancing Dots is a suite of software — plus educational resources and training — that helps people who are blind to read, write and record their music. McCann founded Dancing Dots in 1992.
• Chris Cooke and PlayHymns.com. When Chris Cooke got the Dancing Dots software in 2016, her creativity exploded. She was able to arrange a song and bring it to church and play a duet with a member of the congregation, something she hadn't been able to do before, given the former time and cost of translating music into Braille notation.
• What is Braille Music? Louis Braille, a noted musician, created the Braille musical notation system. Being able to translate music easily between Braille Musical Notation and Western musical notation, and to easily print either of these, helps musicians who are blind share music with other musicians, both sighted and blind, and play music together with ease.
• Musical Instrument Digital Interface and MusicXML. MIDI has made it possible for musicians to play music into their instruments and have it automatically translated into digital musical notation. MusicXML has made universal the file type for a score and allows musicians to share scores across popular music notation applications like Finale or Sebelius.
• The question of parity. James and Chris agree that while Dancing Dots technology has enabled them to take advantage of new musical opportunities, no technology exists that offers them complete parity with sighted musicians because musicians who are blind need additional lead time to get the music scanned correctly or to memorize pieces. Chris adds that preparing the music in a timely fashion and on a budget would help.
• The MIDI-to-brain connection. Bill McCann has explored using the Brainport, a technology from WICAB (which we profiled in an episode from September 2021, on "Training the Brain: Sensory Substitution") to allow musicians who are blind to read music on their tongues. This is important if someone needs to read music live in a performance in order to play any instrument that also requires their hands. Early trials showed signs of success. He posits that maybe someday, maybe soon, people could think new music into notation.

Tweetables:

• "I said, I will never put myself in this position again. If I write something and I am asking other people to play it, and they ask me questions or there's something I am going to know before we meet exactly what I want and what I have." — Bill McCann, founder and president of Dancing Dots
• "So the duet that we played was hot off my printer and went with me and we played it. And it was great to be able to share music in that way because of the technology in the Dancing Dots program." — Chris Cooke, musician and music arranger, creator of Playhymns.com
• "As a blind person, I can say this for myself, often we end up following sighted people, or following somebody. Braille music gives blind musicians the chance to become leaders." — Bill McCann, founder and president of Dancing Dots
• "I've set aside 2023 as a year. I'd really like to kind of develop some more expertise in the area and also come to grips with some of the technology that would help the process." — James Risdon, musician and recorder player
• "Someone sitting there and getting inspired is what we call the MIDI-to-Brain connection. We're not there yet, but you could … think the music in your head … at a computer and … music materializes in the form of a score." — Bill McCann, founder and president of Dancing Dots

Contact Us:

• Contact us at [email protected] with your innovative new technology ideas for people with vision loss.

Pertinent Links:

• Lighthouse Guild
• Dancing Dots
• James Risdon
• Chris Cooke and Playhymns.com

This podcast is about big ideas on how technology is making life better for people with vision loss.

Lots of people have voice-controlled smart home assistants like Siri, Google, or Alexa in their homes…. to listen to the news or to set timers. But they can do so much more! David Frerichs, Principal Engineer, Alexa Experience at Amazon on the aging and accessibility team, shares his design philosophy for making voice assistants more inclusive, and the preferred mode of engagement for every user. He also shares that the next stage of smart home assistants will be ambient computing, where your devices will intuit your needs without you speaking them. We talk with Lighthouse Guild client Aaron Vasquez, who has outfitted his home with smart home technology, and with Matthew Cho, a client who traveled to the Johnson Space Center in Houston to speak to the unmanned Orion Spacecraft via the Amazon Alexa on board, demonstrating that voice assistant technology can bring inclusivity and accessibility to many jobs and industries and are not just for the home anymore.

The Big Takeaways:

• Alexa Onboard the Orion Spacecraft. NASA partnered with Amazon to use the Alexa voice-controlled smart assistant onboard the unmanned Orion spacecraft so that engineers could guide the spacecraft from Mission Control. This project tested the possibility of Alexa for space travel while demonstrating that voice-controlled smart assistants have uses beyond the home. Matthew Cho, an 18-year-old student and client of Lighthouse Guild had the opportunity to travel to Houston to be one of the volunteers to give voice commands to the spacecraft via an Amazon Alexa device while it hurtled through space.
• Accessibility and Preferences. David Frerichs, Principal Engineer, Alexa Experience at Amazon, and someone who works on the aging and accessibility team, has spent his career developing technology that adapts to the ever-changing needs of the user and has cultivated a design philosophy that makes clear that design choices (like voice control) that enable inclusivity for people who are blind can also become the preferred way that most users engage with a device or a tool. Curb cuts are a historical example. David often thinks in terms of "Hot tub safe computing." What can a person do to engage with the device from their hot tub?
• Ambient Computing. David mentions ambient computing, the next phase of smart home technology, in which a network of devices in the built environment intuit and respond to a user's needs without the user even needing to speak a command.
• Smart Homes Today, Smart Industries Tomorrow. Aaron Vasquez is a smart home user. He uses Google Echo to power two smart lamps, operate his smart TV, and control a pet camera to oversee his rambunctious kitties when he's not at home. As a person who is visually impaired, Aaron prefers voice command for running his home in this way. This episode asks how the smart home's tools can be integrated into offices and industries to make these more accessible and inclusive spaces.

Tweetables:

• "To be able to have a rocket be dependent on an AI without anybody having to control the spacecraft is, it is really amazing, and I feel that later on that they'll be able to use it for much more things aside from space." – Matthew Cho, student and client of Lighthouse Guild
• "So their goal was to eventually get different people to be able to go into space … They were trying to see if Alexa would work properly with all sorts of people, normal people. Not just astronauts, like regular, ordinary, everyday people." – Matthew Cho, student and client of Lighthouse Guild
• "You have … permanent need, temporary need, situational need, and preferential need that really can inform us on … how we can address [a] barrier for the particular core use case. But if we do it well, it will serve a much broader community." – David Frerichs, Principal Engineer, Alexa Experience at Amazon
• "We're moving toward … ambient computing. That is … that the system should be able to respond to the needs of the customer, even if the customer doesn't say anything. ... That's … where the boundaries are and where it's gonna continue to be pushed." – David Frerichs, Principal Engineer, Alexa Experience at Amazon
• "We had seen this pack of smart bulbs and they were relatively cheap, and I was like, huh, that's, that's kind of cool. So we were like, you know what? Let's get it. So we got 'em, we hooked them up and that kind of is what started everything." – Aaron Vasquez, smart home user and client of Lighthouse Guild
• "Honestly, it's so much easier if I can ask Google a question and she can come up with the answer, then I'm better off that way instead of actually trying to look it up myself." – Aaron Vasquez, smart home user and client of Lighthouse Guild

Contact Us:

• Contact us at [email protected] with your innovative new technology ideas for people with vision loss.

Pertinent Links:

• Lighthouse Guild
• Alexa in Space
• David Frerichs

This podcast is about big ideas on how technology is making life better for people with vision loss.

Artifacts from Blackbeard's sunken pirate ship are on display in the North Carolina Maritime Museum in Beaufort, North Carolina. But now they are also accessible to visitors who are blind, thanks to the efforts of Peter Crumley, who spearheads the Beaufort Blind Project. In this episode, we ask: How can new technology help make sites like these as accessible to people who are blind as they are to sighted people? We profile three companies applying new technologies paired with smartphone capabilities, to make strides in indoor navigation, orientation, and information transfer. Idan Meir is co-founder of RightHear, which uses Apple's iBeacon technology to make visual signage dynamic and accessible via audio descriptions. We check in with Javier Pita, CEO of the NaviLens QR code technology which we profiled in our first season to see what they have been developing in the last two years. Rather than iBeacons or QR codes, GoodMaps uses LiDAR and geocoding to map the interior of a space. We speak with Mike May, Chief Evangelist. Thanks to Peter Crumley, the North Carolina Maritime Museum is fully outfitted with GoodMaps, and will soon have NaviLens as well. As the prices of these tools come down, the key will be getting them into all the buildings, organizations, and sites of information transfer that people who are blind need to access – which is all of them.

The Big Takeaways:

• Beaufort Blind Project. Peter Crumley, a blind resident of Beaufort, North Carolina, has advocated having accessibility tools brought to various parts of his hometown. Along the way, he helped the North Carolina Maritime Museum outfit itself with GoodMaps technology for indoor navigation, and with NaviLens QR codes for information transfer. Thanks to these new technologies, the museum artifacts are now accessible to everyone.
• RightHear. Idan Meir cofounded RightHear, which uses iBeacon technology paired with users' smartphones to guide visitors who are blind through an indoor space. iBeacons send unique signatures via low Bluetooth signals to phones inside the radius. When these iBeacons are paired with areas of interest in a space (e.g. the front door, the counters, or the bathrooms) users can orient themselves within a space, and identify where they want to go and how they want to navigate to each location. RightHear translates the information embedded in each beacon into audio feedback for users. On the subject of feedback, Idan Meir is looking for beta testers to try out RightHear and provide him with constructive feedback.
• NaviLens. We profiled NaviLens QR code technology in an episode from our first season. In this episode, we follow up with Javier Pita to see what has been in development in the last two years. Since we last spoke, NaviLens has launched NaviLens 360, which uses magnets to help guide users who are blind to the NaviLens codes, even if their camera is having trouble picking up the code, making the app even more user-friendly. In addition, NaviLens has launched a partnership with Kellogg's in Europe and North America to test the effectiveness of the Navilens code on consumer product packaging.
• GoodMaps. GoodMaps uses LiDAR technology to map a space. Lasers are sent out from the LiDAR sensor, and when they bounce back, they have captured distances from the point of origin. Institutions work with GoodMaps to pay for the mapping service, and then users can access the maps for free. The app uses audio to communicate navigational directions with users.
• Technological advancement. Each of these tools relies on component technologies that have gotten less expensive in recent years (iBeacons, QR Codes, and LiDAR). They are also able to exist because their target markets carry smartphones in their pockets, enabling these potential users to access the tools quickly and easily, without much additional hassle or investment.
• Distribution. In this episode, we profile three different approaches to broadening access to indoor navigation technology, including for orientation and information transfer, proving there are many ways to solve these problems. It is good that some of these tools can be paired, as has been done at the North Carolina Maritime Museum, and that users may be able to choose which tools work best for them. The key will be getting them into all the buildings, organizations, and sites of information transfer that people who are blind need to access – which is all of them.

Tweetables:

• "The advocacy is so important; when you're actually interfacing with the app to make the app better and make it work in a way that a blind person really needs it to work." – Peter Crumley, Beaufort Blind Project
• "Well, it's gonna be built from blind perspective philosophy. So not only will it work for me — it will work for anyone, totally blind and fully sighted to give an interactive experience." – Peter Crumley, Beaufort Blind Project
• "Imagine, if this technology will be in all the products, we will solve the problem of accessible packaging for all users." – Javier Pita, NaviLens
• "The point is we have solved the last few yards of the wayfinding problem that is super important for a blind user. And this was born in New York City with the collaboration with the MTA and the department of transportation of New York City." – Javier Pita, NaviLens
• "That camera picks up the environment and it compares it with that point cloud and says, "I see based on this particular image … that you are near the Starbucks," or "You're near Gate 27." –Mike May, GoodMaps
• "It was important and kind of obvious for us from the very early on, you know, that nothing about us without us. It was clear to us that we have to involve users in the process. –Idan Meir, RightHear

Contact Us:

Contact us at [email protected] with your innovative new technology ideas for people with vision loss.

Pertinent Links:

• Lighthouse Guild
• RightHear
• NaviLens
• GoodMaps

This podcast is about big ideas on how technology is making life better for people with vision loss.

In 1997, Gary Kasparov lost an epic chess rematch to IBM's supercomputer Deep Blue, but since then, artificial intelligence has become humanity's life-saving collaborator. This episode explores how AI will revolutionize vision technology and, beyond that, all of medicine.

Karthik Kannan, co-founder of AI vision-tech company Envision, explains the difference between natural intelligence and artificial intelligence by imagining a restaurant recognizer. He describes how he would design the model and train it with positive or negative feedback through multiple "epochs" — the same process he used to build Envision. Envision uses AI to identify the world for a blind or visually-impaired user using only smartphones and smart glasses.

Beyond vision tech, AI enables faster and more effective ophthalmic diagnosis and treatment. Dr. Ranya Habash, CEO of Lifelong Vision and a world-renowned eye surgeon, and her former colleagues at Bascom Palmer, together with Microsoft, built the Multi-Disease Retinal Algorithm, which uses AI to diagnose glaucoma and diabetic retinopathy from just a photograph. She acquired for Bascom Palmer a prototype of the new Kernal device, a wearable headset that records brain wave activity. Doctors use the device to apply algorithms to brainwave activity, in order to stage glaucoma, for example, or identify the most effective treatments for pain.

Finally, AI revolutionizes drug discovery. Christina Cheddar Berk of CNBC reports that thanks to AI, Pfizer developed its COVID-19 treatment, Paxlovid, in just four months. Precision medicine, targeted to a patient's genetic information, is one more way AI will make drugs more effective. These AI-reliant innovations will certainly lower drug costs, but the value to patients of having additional, targeted, and effective therapies will be priceless.

The Big Takeaways:

• Natural vs. artificial intelligence, and the "restaurant recognizer." Karthik Kannan, CEO and co-founder of Envision explains the difference between natural and artificial intelligence by describing how humans recognize restaurants in a foreign city and comparing that to how he'd train a "restaurant recognizing algorithm." Here's a hint: the algorithm needs a lot more data.
• Sensor fusion AI. AI developers are interested in using different types of sensors together to give the algorithms a sense of the world closer to human intelligence. One example is the use of LiDAR in the Envision app, in addition to the phone camera.
• Transhumanism. Humans don't have LiDAR. Does that mean AI will surpass human capability? Karthik offers that some radiology AI have higher accuracy than human radiologists, but he thinks it will be much more of a partnership between the human and the machine.
• Multi-Disease Retinal Algorithm. Dr. Ranya Habash and her colleagues at Bascom Palmer Eye Institute worked with Microsoft on an AI diagnostic tool. They fed the algorithm 86,000 images of eyes, labeled with relevant diseases, and taught the machine to diagnose eye disease with just a photograph, making remote diagnosis not just possible but inexpensive.
• The Brain-Machine Interface. Dr. Habash wrote a grant that earned Bascom Palmer a prototype of the Kernal device, a helmet-like device that measures brainwave activity. Doctors used this device to create a "brain-machine interface" which advances brain research on glaucoma, diabetic retinopathy, Alzheimer's, and pain management.
• Bias in AI. Karthik Kannan reminds us that the biggest threat that humanity faces from AI is bias encoded in the algorithms. This is a real harm that humans have already experienced, and AI engineers need to be extremely sensitive to ensure they are not encoding their own biases.
• AI for Drug Discovery. Christina Cheddar Berk, a reporter for CNBC, shares how the pace of drug discovery is set to speed up, thanks to AI algorithms and supercomputing power that can cycle through millions of possible chemical compounds per second to I.D. effective options. Pfizer used a similar process to develop Paxlovid, in a process that took only four months.

Tweetables:

• "The secret sauce is always in the data." — Karthik Kannan, CEO and Co-Founder of Envision
• "Human intelligence is so holistic. We have so many sensors on our bodies. […] Whereas an AI is taught only images." — Karthik Kannan, CEO and Co-Founder of Envision
• "I know what's going to work and what's not going to work within thirty seconds of seeing it. […] They need to show up with a smartphone. Then I'll take them seriously." — Dr. Ranya Habash, CEO, Lifelong Vision
• "I don't think there's anything more powerful in medicine than to be able to treat a patient and get rid of a problem that is plaguing them so much." — Dr. Ranya Habash, CEO, Lifelong Vision
• "If you can measure it you can control it." — Dr. Ranya Habash, CEO, Lifelong Vision
• "It strongly takes over the bias of whoever is actually feeding the data […] and I think that has much, much more potential for harm than an AI taking over humanity." — Karthik Kannan, CEO and Co-Founder of Envision
• "The value for patients of having those additional therapies available; it's hard to put a price on." — Christina Cheddar Berk, reporter, CNBC

Contact Us:

• Contact us at [email protected] with your innovative new technology ideas for people with vision loss.

Pertinent Links:

• Lighthouse Guild
• Karthik Kannan
• Dr. Ranya Habash
• Zephin Livingston
• Christina Cheddar Berk

This podcast is about big ideas on how technology is making life better for people with vision loss.

Innovations in implant technology are advancing at lightning speed, profoundly impacting the lives of people who are blind or visually impaired. In On Tech And Vision, we've profiled some amazing new implant technologies that have the potential to restore people's sight. But in this episode, we pump the breaks — because we need to address a critical part of the innovation process: the ethical frameworks that protect participants in early clinical trials, and the need for an updated framework that ensures patient protections without stifling innovation and development.

Discussions between doctors and participants in clinical trials almost always focus on the new technology and very rarely on the manufacturer who sponsors the clinical trial — and almost never on the long-term commitment and financial viability of the company sponsoring the technology. And while clinical trial informed consent includes whose responsibility it is to remove the implants should they fail during the trial, that responsibility usually ends once the trial is over. At that stage, who will maintain or remove the implants that are still housed in patients' bodies?

In this episode, we talk about innovative implants such as the Argus II, which we featured in the first season of On Tech And Vision. The Argus II is a microchip implanted under the retina that, in combination with a special headset, provided some vision to people who otherwise had none. And while the technology was exciting, the company discontinued the retinal implant three years ago, and the Argus II was eventually sold to another pharmaceutical company.

Dr. Joseph Fins, Professor of Medical Ethics and Professor of Medicine at Weill Cornell Medical Center in New York, joins us to share his thoughts on today's big idea: How do we balance the life-changing potential of electroceutical implant technology with the ethics of caring for early participants — particularly after clinical trials are over?

The Big Takeaways:

• Examples of electroceutical implants. Cochlear implants, retinal implants, and deep brain stimulators are examples of scientific advances that rely on in-dwelling devices.
• Regulatory framework today. The relationships between researchers and clinical trial participants are regulated by institutional review boards, which came out of the National Research Act of 1974. However, while this framework works well for drug trials, new issues specific to implants need to be addressed by new regulations. For example, who is responsible for people left with in-dwelling devices once trials are over? If the sponsoring company no longer supports their devices, are they victims of abandonment? Are the timelines for drug trial success too short to be relevant for implant device trials, since it may take the body longer to adopt a new technology than to respond to a new drug?
• Ancillary care obligations. Henry Richardson, in his book Moral Entanglements: The Ancillary Care Obligations of Medical Researchers writes that historically, researchers — to avoid conflicts of interest — did not assume a clinical care role. However, that is changing, as researchers realize they have an obligation to share actionable results with patients. The result is that there is even less of a "bright line distinction," as Dr. Fins says, "between research and therapy."
• Collective responsibility. Who is responsible for the long-term well-being of participants in electroceutical trials? Dr. Fins suggests that the sponsoring company, the medical school where the research is taking place, and the government should share responsibility. It's a collective problem, he says.
• Some solutions. Requiring researchers, sponsoring companies, and researching universities to include in the costs of development insurance to cover long-term care for participants is one potential solution that Dr. Fins imagines. He also offers that researchers and sponsoring companies that develop successful and adopted medical products could subsidize the field. Or, he suggests, a tax on gaming devices (adjacent to electroceutical implants) to sustain people who are given indwelling devices in clinical trials.
• The law. The law needs to evolve to address the specific vulnerabilities of participants in electroceutical implant trials. Dr. Fins suggests that there are provisions within the Americans with Disabilities Act that account for assistive technologies that were relevant when the act was written in the nineties. According to Dr. Fins, these provisions in that could be read with a more contemporary lens, to include the assistive technologies of today (which would encompass electroceutical implants). There is room for lawyers and legal scholars to impact the legal frameworks in place now, to expand coverage from the ADA to protect participants in clinical trials for electroceutical implants.
• "Victims of Our Own Success." Electroceutical implants are a miracle, says Dr. Fins. They are human ingenuity at its best. The science is harder to solve than the bureaucracy, but the bureaucracy to sustain medical advancements like these must catch up, or, ultimately, the vulnerability of trial participants threatens to impede scientific progress.
• Danger to the field. Clinical trials rely on willing participants, and when participants are not supported after trials end, it erodes participants' trust across the field. Without a clear set of protections in place for participants in clinical trials, scientific and medical advancement in the area of electroceutical implants may be impeded.

Tweetables:

• "I think this is a perfect rationale for insurance."— Dr. Joseph Fins, Weill Cornell Medical Center
• "This is a huge problem. … We're victims of our own success."— Dr. Joseph Fins, Weill Cornell Medical Center
• "It's human ingenuity at its very best. And the fact that we can't figure out the bureaucracy to sustain this? … The science is harder than the politics and the bureaucracy, but we're being overmatched by the politics and the bureaucracy." — Dr. Joseph Fins, Weill Cornell Medical Center
• "When stories like this come out it makes recruitment very hard." — Dr. Joseph Fins, Weill Cornell Medical Center
• "These retinal implants, these deep brain stimulators, … they're gonna be looked upon as primitive halfway technologies 50 and 100 years from now. But we're only gonna get there if we're able to do this research." — Dr. Joseph Fins, Weill Cornell Medical Center
• "Once you understand these facts, the ethics are pristine. They're clear." — Dr. Joseph Fins, Weill Cornell Medical Center

Contact Us:

• Contact us at [email protected] with your innovative new technology ideas for people with vision loss.

Pertinent Links:

• Lighthouse Guild
• Dr. Joseph Fins

Listener nominations are open from July 1st - July 31, 2021. On July 1st visit: https://www.podcastawards.com, sign-up and vote

This podcast is about big ideas on how technology is making life better for people with vision loss.

People who are blind or visually impaired know all too well the challenges of living in a sighted world. But today, the capabilities of computer vision and other tech are converging with the needs of people who are blind and low-vision and may help level the playing field for young people with all different sensory abilities. These tools can pave the way for children's active participation and collaboration in school, in social situations, and eventually, in the workplace, facilitating the important contributions they will make to our world in their adult lives.

Access to educational materials is a consistent challenge for students and adults who are blind, but Greg Stilson, the head of Global Innovation at American Printing House for the Blind (APH), is trying to change that. Together with partner organizations Dot Inc. and Humanware, APH is on the verge of being able to deliver the "Holy Braille" of braille readers, a dynamic tactile device that delivers both Braille and tactile graphics in an instant, poised to fill a much-needed gap in the Braille textbook market. Extensive user testing means the device is as useful for people who are blind as possible. Greg sees a future in which more inclusively designed and accessible video games, augmented reality (AR), and virtual reality (VR) will help children who are blind learn with greater ease, and better engage with their sighted peers.

Enter Dr. Cecily Morrison, principal researcher at Microsoft Research in Cambridge, UK. Based on extensive research and co-designing with people who are blind, she and her team developed PeopleLens, smart glasses worn on the forehead that can identify the person whom the user is facing, giving the user a spatial map in their mind of where classmates (as one example) are in space. PeopleLens helps children who are blind overcome social inhibitions and engage with classmates and peers, a skill that will be crucial to their development, and in their lives, as they move into the cooperative workspaces of the future.

The Big Takeaways:

• Robin Akselrud, an occupational therapist and assistant professor at Long Island University in Brooklyn, author of MY OT Journey Planner and The My OT Journey Podcast, explains how a baby who is born blind becomes inhibited from their first developmental milestones. She explains the stressors that these children might face upon attending school and describes the kinds of interventions that occupational therapy offers.
• Bryce Weiler, disability consultant, sports enthusiast, and co-founder of the Beautiful Lives Project, emphasizes how important it is for children who are blind or low-vision to have rich sensory experiences — and life experiences — which give them a chance to flourish and socialize with peers. Beautiful Lives Project offers opportunities to do that.
• Greg Stilson, Director of Global Innovation at American Printing House for the Blind, and his team are developing a dynamic tactile device (DTD) that can switch seamlessly between Braille and tactile graphics — the "Holy Braille" of braille devices. The DTD is made possible by developments in pin technology by Dot Inc, and APH. Humanware developed the software for the device. No longer using the piezoelectric effect to move pins has reduced the cost of the device significantly, and APH can funnel federal funds to reduce the price further, making the DTD a potential, viable option for institutions.
• Cecily Morrison, principal researcher at Microsoft Research in Cambridge UK, and her team developed PeopleLens, a head-worn pair of smart glasses that lets the wearer know who is in their immediate vicinity. Dr. Morrison and her team tested it in classrooms for school-age children who are blind or visually impaired and found that PeopleLens reduces students' cognitive load and helps young people overcome social anxiety and inhibitions that Robin Akselrud described at the top of the show. Wearers of PeopleLens learn to develop mental models of where people are in a room, and gain the confidence to engage others, or not, as they choose. Once social skills are built, students no longer have to wear the device but are set up for more successful social interactions at school and in their lives to come.

Tweetables:

• If they have a visual impairment, it really impacts them from early on, from that first development milestone. — Robin Akselrud, occupational therapist and assistant professor at Long Island University in Brooklyn, author of MY OT Journey Planner and The My OT Journey Podcast
• For children, just giving them that foundation of making friendships as they're growing up, and the opportunity to be a part of something, sport can allow them to do that, and it also gives them the chance to do things that their peers are taking part in. —Bryce Weiler, disability consultant, sports enthusiast, and co-founder of the Beautiful Lives Project
• This was what the field regards as the "Holy Braille" right? Having both [Braille and tactile graphics] on the same surface. —Greg Stilson, Director of Global Innovation at American Printing House for the Blind
• With the advancements of virtual reality and augmented reality, … along with the idea of making experiences and video games and things like that more inclusive, it's going to create a more inclusive way for blind kids to engage with their sighted peers. — Greg Stilson, Director of Global Innovation at American Printing House for the Blind
• We found that "people" was the thing that was most interesting to people. And that doesn't surprise us. We are people, and we like other people. — Dr. Cecily Morrison, principal researcher at Microsoft Research in Cambridge UK
• They can go out and find someone that they want to play with. They can choose not to talk to somebody by turning their head away from them, and the moment that they understand the agency they have in those situations is when we see a significant change in their ability to place people and to engage with them. — Dr. Cecily Morrison, principal researcher at Microsoft Research in Cambridge UK
• When we look at the workplaces of today, they're often very collaborative places. So you can be the best mathematician in the world, but if you struggle to collaborate, you're not building the AI technologies of tomorrow. By helping kids ensure that they have a strong foundation in these attentional skills, we're giving them a significant lift. — Dr. Cecily Morrison, principal researcher at Microsoft Research in Cambridge UK

Contact Us:

Contact us at podcasts@lighthouseguild.org with your innovative new technology ideas for people with vision loss.

Pertinent Links:

• Lighthouse Guild
• Robin Akselrud
• Bryce Weiler
• Greg Stilson
• Dr. Cecily Morrison