Let's Talk Risk! Podcast

Summary

“We are all at some point, either patients or loved ones of patients. So, one way or another, we are all touched by the medical technology we create ourselves.

In this episode of the Let's Talk Risk Podcast, Bijan Elahi helps us appreciate the most important aspect of our work as risk practitioners, engineers and quality/regulatory affairs professionals in the medical device industry. That our work makes a direct impact on health and wellness of our loved ones and patients across the world.

We focus on 4 big themes in risk management that will continue to demand our attention in 2025 and beyond - AI/ML, cybersecurity, post-market surveillance and usability engineering. QA/RA professionals will need to stay updated as they help their organizations navigate through challenges in these areas.

We also talk about an upcoming Conference in April 2025 that will focus exclusively on safety risk management of medical devices. Register now - early bird discounted pricing available until 28 February 2025.

Listen to the full 30-minute podcast or jump to a section of interest listed below.

Chapters

00:00 Introduction and Keynote Insights

06:13 AI and Machine Learning in Medical Devices

09:08 Cybersecurity Challenges in Medical Devices

12:19 Post-Market Safety Surveillance

16:07 Usability Engineering and Human Factors

18:33 Upcoming Conference Details and Takeaways

Suggested links:

* International conference on medical device safety risk management.

* LTR: AI/ML in MedTech.

* LTR: Medical device cybersecurity now more critical than ever.

* LTR: Best practices for applying usability engineering to medical devices.

Key Takeaways

* We are all at some point either patients or loved ones.

* The impact of biomedical engineers on MedTech safety is huge.

* AI/ML devices evolve and their performance is not static.

* Cybersecurity is increasingly critical for medical devices.

* Post-market surveillance is essential for ongoing safety.

* Usability engineering can prevent misuse of medical devices.

* Collaboration across functions is vital for effective risk management.

* The FDA emphasizes explainability in AI ML devices.

* Learning from post-market data can improve future designs.

* Conferences provide valuable opportunities for knowledge sharing.

Keywords

Risk management, AI, machine learning, cybersecurity, medical devices, usability engineering, post-market surveillance, safety conference, safety, healthcare

About Bijan Elahi

Bijan Elahi is the founder of MedTech Safety and a globally recognized expert on safety risk management of medical devices. Bijan is a technical fellow, and an affiliate professor at Drexel University, USA. Bijan is also a lecturer at Eindhoven University of Technology in the Netherlands, where he teaches risk management to doctoral students in engineering. He is a frequently invited speaker at international professional conferences, and is also a contributor to ISO 14971, the international standard on the application of risk management to medical devices. He has trained over 10,000 people worldwide and his book Safety Risk Management of Medical Devices, now in its second edition, is a bestseller.

Disclaimer

Information and insights presented in this podcast are for educational purposes only. Views expressed by all speakers are their own and do not reflect those of their respective organizations.

Let's Talk Risk! is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

“There is a clear case to put the money into the design of the product, and control that, than relying on the human factor.”

In this Let’s Talk Risk! conversation, Alex Saegert highlights the importance of focusing on risk reduction through safety by design, rather than relying on information for safety to users. He recommends risk practitioners to apply best practices from systems safety to build a clear understanding of system-level hazards that lead to harm in a hazardous situation through a sequence of events. Understanding the sequence of events can help create a more robust design to minimize the probability of occurrence of one or more elements involved.

He reminds us that there are tools available that can be applied without significantly increasing complexity of the design. Additionally, he recommends linking hazard analysis to design controls (DFMEA) and process controls (PFMEA) to reduce and control probabilities of occurrence of the events that contribute to hazardous situations, while maintaining traceability between hazards and control actions.

Listen to this Let’s Talk Risk! conversation with Alex Saegert, which also includes an open discussion with a live audience. Jump to a section of interest using these timestamps.

00:01:30 Introduction

00:02:30 Connecting risk analysis with design requirements for a balloon catheter

00:07:10 Estimating probability of occurrence for sequence of events

00:10:03 Connecting design with process FMEA for improved reliability

00:14:00 Applying functional safety to medical device design/development

00:19:10 Improving cross-functional collaboration

00:20:42 Open discussion and audience Q&A

00:31:20 Closing comments and key takeaways

If you enjoyed this podcast, consider subscribing to the Let’s Talk Risk! newsletter.

Suggested links:

LTR: Tips for integrating risk analysis with design controls

LTR: Design your medical device to be robust to use conditions

LTR: Overcoming challenges in linking risk management with design controls

About Alex Saegert

Alex Saegert is currently the President of Saegert Solutions Inc., an engineering consultancy focused on new technology commercialization, ensuring safety, reliability and usability. His professional experience includes developing ISO14971 compliant risk management programs for manufacturers of advanced class III medical devices. With over 25 years' experience in reliability engineering, product development and failure analysis across a variety of industries, Alex is a professional engineer, an ASQ-Certified Reliability Engineer (CRE), and IEC61508 Functional Safety Engineer.

Let’s Talk Risk! with Dr. Naveen Agarwal is a weekly live audio event on LinkedIn, where we talk about risk management related topics in a casual, informal way. Join us at 11:00 am EST every Friday on LinkedIn.

Disclaimer

Information and insights presented in this podcast are for educational purposes only. Views expressed by all speakers are their own and do not reflect those of their respective organizations.

“Overconfidence in the performance of AI models in the development phase is a common problem.”

In this Let’s Talk Risk! conversation, David Giese highlights the growing need for a rigorous, disciplined development process with the goal of meeting regulatory requirements for marketing authorization. He shares how AI developers are often over-confident in the performance of their models, but disappointed when these models are subjected to rigorous scrutiny during the regulatory review.

Software as a Medical Device (SaMD), including AI/ML devices, are growing exponentially in MedTech. Rapidly evolving technology offers an opportunity to develop innovative products, but it also presents new risks. Security and Cybersecurity concerns are on the rise, as well as, concerns about transparency and explainability of AI/ML models.

David points to new guidance documents from the FDA and emerging standards. As an example, software developers should consider using IEC 81001-5-1 - Health software and health IT system safety, effectiveness and security - along with IEC 62304 - Medical device software - Software life cycle processes.

Listen to this Let’s Talk Risk! conversation with David Giese, which also includes an open discussion with a live audience. Jump to a section of interest using these timestamps.

00:01:30 Introduction

00:02:30 Current state of AI/ML applications in MedTech

00:04:05 Different types of AI models - adaptive vs. generative AI

00:06:25 Challenges in meeting regulatory expectations for AI in MedTech

00:08:18 Use of synthetic data in AI model development

00:09:30 Best practices for AI development and regulatory submissions

00:12:02 Cybersecurity challenges for SaMD and AI/ML devices

00:14:03 New skills risk practitioners need to develop

00:15:09 Open discussion and audience Q&A

00:27:20 Closing comments and key takeaways

If you enjoyed this podcast, consider subscribing to the Let’s Talk Risk! newsletter.

Suggested links:

LTR: EU vs. FDA - Aligned but different in approach to safety of AI/ML devices

LTR: A new paradigm for building connected medical devices

FDA: Cybersecurity in Medical Devices: Quality System Considerations and Content of Premarket Submissions, Issued September 2023.

About J. David Giese

J. David Giese is currently President and Partner at Innolitics, a consulting firm specializing in development of new medical software and regulatory submissions to the FDA. His team has helped develop and receive FDA clearance for over 60 medical devices, both SaMD and SiMD, over the last 12 years. His expertise include application development, custom software development, engineering design, project management, user experience design, cybersecurity, software testing and technical writing. He holds Bachelors and Masters degrees in Biomedical Engineering.

Let’s Talk Risk! with Dr. Naveen Agarwal is a weekly live audio event on LinkedIn, where we talk about risk management related topics in a casual, informal way. Join us at 11:00 am EST every Friday on LinkedIn.

Disclaimer

Information and insights presented in this podcast are for educational purposes only. Views expressed by all speakers are their own and do not reflect those of their respective organizations.

Summary

“Medical devices are now increasingly connected in a hospital network. Or even if they are not, they are vulnerable to cyber attacks”.

In this episode of the Let's Talk Risk Podcast, Eric Henry highlights the growing concern about security and cybersecurity of medical devices. As technology evolves and medical devices increasingly operate in an i…

“Software more than any other type of device, has integrations with other systems and devices. You have to consider a complex network of dependencies and infrastructure when analyzing risks.”

In this Let’s Talk Risk! conversation, Megan Kane highlights the growing role of software, including artificial intelligence, in in-vitro diagnostic devices, including in next generation sequencing applications. We are now living at at time where technology is rapidly enabling precision medicine solutions for an individual patient.

In this high-tech environment, increasingly driven by software, new risks are also emerging. Cybersecurity, patient data privacy, service availability are common concerns. But when it comes to IVDs, it becomes challenging to connect the dots to find a direct link to the patient. Inaccurate or delayed results are common risks related to IVDs, but their potential impact on a patient is difficult to figure out with sufficient confidence. It often results in underestimating the severity of potential harm as reflected by a recent FDA warning letter.

Another key development in this area is the new FDA regulation for Lab Developed Tests (LDT), which is expected to add new requirements for pre-market reviews, risk management and quality system.

Listen to this Let’s Talk Risk! conversation with Megan Kane, which also includes an open discussion with a live audience. Jump to a section of interest using these timestamps.

00:01:10 Introduction

00:01:50 How software is increasingly driving IVD applications

00:03:30 Emerging risks with software in IVDs

00:06:15 Evolving regulatory approach to IVDs affecting LDTs

00:09:13 Increasing focus on risk management for LDTs

00:12:50 Risk considerations when software is a component of an IVD

00:15:30 Open discussion and audience Q&A

00:28:01 Closing comments and key takeaways

If you enjoyed this podcast, consider subscribing to the Let’s Talk Risk! newsletter.

Suggested links:

LTR: 3 pillars for defining your IVD risk management strategy

LTR: Focus on intended use for risk management of IVDs

FDA: Laboratory developed tests

About Megan Kane

Megan Kane is currently Director of Regulatory Operations at Velsera, where she supports the mission of radically improving healthcare globally through multi-omics to fuel growth of precision medicine. She is primarily involved in developing advanced software, including AI applications for end-to-end Next Gen Sequencing analysis. Previously she held various roles in Quality and Regulatory functions at both large and small medical device and in-vitro diagnostic companies. She holds a Bachelor’s degree in Molecular and Cell Biology, and Internal Auditor certifications from BSI in ISO 13485:2016 and ISO 9001:2015.

Let’s Talk Risk! with Dr. Naveen Agarwal is a weekly live audio event on LinkedIn, where we talk about risk management related topics in a casual, informal way. Join us at 11:00 am EST every Friday on LinkedIn.

Disclaimer

Information and insights presented in this podcast are for educational purposes only. Views expressed by all speakers are their own and do not reflect those of their respective organizations.

Risk practitioners in MedTech now have a unique opportunity to help accelerate development and post-market improvement of AI/ML enabled medical devices while also reducing the regulatory burden.

PCCP, or Pre-Determined Change Control Plan, is a novel regulatory approach that allows device manufacturers to include planned changes in their initial submission and avoid additional submissions in the post-market phase when these changes are implemented.

This is even more important for AI/ML enabled medical devices, which are expected to learn from new data in real-world applications to improve both user experience and performance without compromising patient safety.

As an example, Apple recently utilized a tailored PCCP approach in their regulatory submissions to receive advance clearance for their planned post-market changes. In case of the Hearing Aid Function, they focused on features to improve user experience. In the Sleep Apnea Notification function (SANF), on the other hand, they focused on improving the performance of the core AI model based on real-world data. In both cases, they proposed a rigorous verification and validation plan, and labeling updates.

Risk managers can play a key role in planning these modifications using a risk-based approach. They can help develop testing and implementation protocols. Finally, they can help articulate the overall benefits and risks to convince the FDA that these planned changes would improve safety and effectiveness.

Risk practitioners therefore have an opportunity to become more strategic and influential by playing a proactive role. They can collaborate with regulatory professionals, product developers and clinicians right from the start of product development.

Listen to the 22-minute audio summary above that covers the following topics:

* Key differences between FDA’s draft and final PCCP guidance

* Lessons learned from Apple’s tailored approach to PCCP

* Understanding benefits and risks of PCCPs

* Opportunities for risk practitioners

* Challenges in implementing PCCPs

Disclaimer

This article was prepared with the help of Google NotebookLM, an artificial intelligence research assistant, using the following sources:

* FDA Draft and Final Guidance: PCCP for AI/ML enabled devices.

* Apple De Novo: Hearing Aid Feature, Decision Summary.

* Apple 510k: Sleep Apnea Notification Feature (SANF).

* Caption Health De Novo: Automated Ejection Fraction Software.

* Notes created using Google NotebookLM in response to user prompts.

All output(s), including the audio summary, were reviewed by a human for accuracy and relevance. This article is intended for educational purposes only and should not be considered as regulatory advice.

If you liked this post, consider becoming a free or paid subscriber to Let’s Talk Risk!.

Summary

“You are not gong to lose your job to AI, but you may lose your job to someone who knows how to use AI better than you do”.

In this episode of the Let's Talk Risk Podcast, we explore the benefits of applying Generative AI to improve consistency of quality system documentation in MedTech with Garth Conrad, VP of Quality at Flex Health Solutions.

G…

Applications of AI in medical devices are growing rapidly. The regulatory environment in both the US and the EU is also changing fast. In this dynamic environment, it is important to stay updated and practice a flexible approach to both risk management and your regulatory strategy.

Listen to a brief audio summary above, about the emerging regulatory environment in these two major jurisdictions, and key takeaways for risk practitioners and regulatory professionals.

Regulatory environment is changing rapidly but there is new guidance

There is good news! A new guidance document in the form of a questionnaire was recently published by the Team-NB, the European Association of Medical devices Notified Bodies. The joint Team-NB/IG-NB Questionnaire on Artificial Intelligence in Medical Devices offers device manufacturers a process-oriented roadmap to demonstrate conformity to the EU-MDR (or EU-IVDR) requirements.

The term risk(s) appears 50 times in this questionnaire, highlighting the significance of risk management as a critical factor in ensuring safety and effectiveness of AI devices throughout their lifecycle. Out of a total of 189 questions across 26 categories, 32 (17%) are explicitly related to risk management!

FDA’s regulatory approach is considerably less prescriptive and more collaborative. The regulatory framework for the pre-market review is no different for AI-enabled devices compared to medical devices in general, including Software as a Medical Device (SaMD). A majority of nearly 1000 AI/ML enabled devices have been authorized as Class II devices, either through the De Novo, or the 510k pathway. The most important requirement is to demonstrate safety and effectiveness through valid scientific evidence that benefits of the intended use outweigh probable risks.

Let us take a closer look at the emerging regulatory environment in the US and EU

First, there is broad alignment at a high level between FDA and the EU

At a high level both FDA and notified bodies are generally aligned on the need to demonstrate safety and effectiveness of AI-enabled medical devices. Here are 3 specific areas of convergence in these two :

1. Focus on safety and effectiveness

Both the FDA and the European approach, reflected by Team-NB, prioritize patient safety and the effectiveness of AI-enabled medical devices.

2. Recognition of AI’s unique challenges

Both recognize that AI presents unique regulatory challenges due to its complexity, iterative nature, and reliance on data.

3. Importance of real-world monitoring

Both emphasize the need for ongoing monitoring of AI-enabled devices in real-world settings to ensure safety and performance.

Second, the Team-NB approach focuses on certifiability using a process-oriented questionnaire

1. Process-oriented approach for safety

The European approach, as evidenced by the questionnaire, focuses on ensuring the safety of AI-based medical devices through a comprehensive evaluation of processes throughout the device lifecycle.

2. Detailed requirements and documentation

The questionnaire outlines specific requirements for documentation, competence of development teams, risk management, data management, model development, and post-market surveillance.

3. Emphasis on certifiability

The questionnaire highlights the challenges of certifying AI-based medical devices, particularly those with self-learning capabilities, and emphasizes the need for robust validation processes.

4. Consideration of AI-specific security risks

The questionnaire addresses AI-specific cybersecurity risks like adversarial attacks and emphasizes the importance of security lifecycle management.

Finally, FDA’s approach is more collaborative and adaptive

1. Collaborative and adaptive

The FDA emphasizes collaboration with stakeholders (developers, patients, academia, global regulators) and a commitment to adapt regulations to the rapidly evolving AI landscape.

2. Focus on bias mitigation and health equity

The FDA prioritizes addressing bias in AI algorithms and promoting health equity by ensuring data representativeness.

3. Emphasis on lifecycle management

The FDA stresses the importance of managing AI applications throughout the medical product lifecycle, from design to deployment, monitoring, and maintenance.

4. Commitment to guidance and regulatory science

The FDA is actively developing guidance documents and supporting research to address the unique challenges of evaluating and regulating AI in medical products.

Key takeaways for risk practitioners and regulatory professionals

In this rapidly changing environment, it is very important for risk practitioners and regulatory professionals to stay current with evolving regulatory approaches. Here are 3 key takeaways to keep in mind:

1. Practice a flexible and adaptable approach to risk management

Risk practitioners and regulatory professionals need to stay informed of the latest developments and adjust their practices accordingly. They must also anticipate future changes and build flexibility into their risk management frameworks and compliance strategies.

2. Understand and address bias in AI systems

Identifying and quantifying bias in AI systems can be complex. Risk practitioners and regulatory professionals need to develop robust methodologies for assessing bias and its potential impact on patient safety and health equity. This includes understanding the sources of bias in training data, evaluating the fairness of AI algorithms, and implementing strategies for monitoring and mitigating bias in deployed systems.

3. Apply a tailored approach to address regulatory concerns in each market

The FDA is primarily focused on the end product and its intended use, while the EU is taking a more process-oriented approach that emphasizes the entire AI lifecycle. These differing approaches may lead to varying risk profiles and require adjustments to risk management strategies depending on the target market. Risk practitioners and regulatory professionals need to carefully consider these differences and develop tailored strategies that meet the specific requirements of each jurisdiction.

In conclusion

It is clear that AI applications in MedTech are going to continue growing. We are still in the early phase of AI applications, especially in healthcare.

At the same time, the regulatory environment is evolving rapidly. Both the FDA and the EU are moving fast to catch up with technology. While a focus on safety and effectiveness remains as the centerpiece of the regulatory approach, there are distinct differences in these two major jurisdictions. Good news is that new guidance from these regulators is coming out to clarify their latest thinking.

Risk management is an essential aspect of regulatory focus. There are new and emerging concerns about risks associated with AI/ML devices. Risk practitioners and regulatory professionals must stay current, and develop flexible, adaptable and tailored strategies to respond to this dynamic regulatory environment.

Disclaimer

This article was prepared with the help of Google NotebookLM, an artificial intelligence enabled research assistant, using the following sources:

* FDA white paper: Artificial Intelligence & Medical Products.

* Team-NB - Questionnaire: Artificial Intelligence in Medical Devices.

* FDA: Good Machine Learning Practice for Medical Device Development.

* IMDRF: Good machine learning practice for medical device development.

* Notes created using Google NotebookLM in response to user prompts

All output(s), including the audio summary, were reviewed by a human for accuracy and relevance. This article is intended for educational purposes only and should not be considered as regulatory advice.

If you liked this post, consider becoming a free or paid subscriber to Let’s Talk Risk!.

“The classic thing with AI is that the hard stuff is easy and the easy stuff is hard. It can do math I cannot do, but it cannot do the reasoning I find easy.”

In this Let’s Talk Risk! conversation, we discuss key challenges and opportunities for applying Artificial Intelligence/Machine Learning (AI/ML) in MedTech. This was an open conversation with a live audience as part of the weekly Let’s Talk Risk! conversation on LinkedIn.

AI/ML applications in MedTech are growing rapidly. FDA has authorized nearly 1000 such applications, and this trend is only expected to grow. Our conversation included a variety of topics about this rapidly evolving field.

This discussion involved comments from Emanuel Tkach, MD, Bijan Elahi, Edwin Bills, Rafael Pozos, Wag Hanna, Phil Deming, Andy David and Ritam Priya.

Jump to a section of interest using these timestamps.

00:03:30 Key factors related to AI/ML applications in MedTech

00:05:30 Dynamic nature of AI/ML causing performance drift

00:07:30 Upcoming ISO guidance on risk considerations for AI/ML applications

00:09:00 Keeping the human in the loop

00:10:25 Data quality issues and best practices for AI/ML

00:12:17 Cybersecurity considerations affecting safety

00:14:20 Lessons learned from clinical evaluation of conventional devices

00:16:25 Is agile software development for AI/ML too slow?

00:19:12 Treating AI/ML as a tool and a team member, and its limitations

00:23:30 A few examples of AI/ML applications in MedTech

00:21:35 Watch out for human over-reliance on AI/ML

00:27:44 Experience with ChatGPT prompts

00:32:22 Closing comments and key takeaways

If you enjoyed this podcast, consider subscribing to the Let’s Talk Risk! newsletter.

Suggested links:

LTR: AI/ML in MedTech

FDA: QA/RA aspects of AI/ML devices

Disclaimer

Information and insights presented in this podcast are for educational purposes only. Views expressed by all speakers are their own and do not reflect those of their respective organizations.

Summary

“Apply the usability engineering process diligently, and with the spirit of inquiry. You will learn things that you never thought. And these are the things that make your device possible”.

Human factors engineering, or usability engineering, is often thought of as a check-the-box activity for the purpose of regulatory compliance. But it is a lot …