The MLSecOps Podcast

The MLSecOps Podcast

By MLSecOps.comTechnology
Download on the App Store

The MLSecOps Podcast episodes

  • A Holistic Approach to Understanding the AI Lifecycle and Securing ML Systems: Protecting AI Through People, Processes & Technology; With Guest: Rob van der Veer

    Send a text

    Joining us for the first time as a guest host is Protect AI’s CEO and founder, Ian Swanson. Ian is joined this week by Rob van der Veer, a pioneer in AI and security. Rob gave a presentation at Global AppSec Dublin earlier this year called “Attacking and Protecting Artificial Intelligence” which was a large inspiration for this episode. In it, Rob talks about the lack of security considerations and processes in AI production systems compared to traditional software development, and the unique challenges and particularities of building security into AI and machine learning systems.

    Together in this episode, Ian and Rob dive into things like practical threats to ML systems, the transition from MLOps to MLSecOps, the [upcoming] ISO 5338 standard on AI engineering, and what organizations can do if they are looking to mature their AI/ML security practices.

    This is a great dialogue and exchange of ideas overall between two super knowledgeable people in this industry. So thank you so much to Ian and to Rob for joining us on The MLSecOps Podcast this week.


    Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com.

    Additional tools and resources to check out:
    Protect AI Guardian: Zero Trust for ML Models

    Recon: Automated Red Teaming for GenAI

    Protect AI’s ML Security-Focused Open Source Tools

    LLM Guard Open Source Security Toolkit for LLM Interactions

    Huntr - The World's First AI/Machine Learning Bug Bounty Platform

    30 min
  • ML Model Fairness: Measuring and Mitigating Algorithmic Disparities; With Guest: Nick Schmidt

    Send a text

    This week we’re talking about the role of fairness in AI/ML. It is becoming increasingly apparent that incorporating fairness into our AI systems and machine learning models while mitigating bias and potential harms is a critical challenge. Not only that, it’s a challenge that demands a collective effort to ensure the responsible, secure, and equitable development of AI and machine learning systems.

    But what does this actually mean in practice? To find out, we spoke with Nick Schmidt, the Chief Technology and Innovation Officer at SolasAI. In this week’s episode, Nick reviews some key principles related to model governance and fairness, from things like accountability and ownership all the way to model deployment and monitoring.

    He also discusses real life examples of when machine learning algorithms have demonstrated bias and disparity, along with how those outcomes could be harmful to individuals or groups. 

    Later in the episode, Nick offers some insightful advice for organizations who are assessing their AI security risk related to algorithmic disparities and unfair models.


    Additional tools and resources to check out:
    AI Radar
    ModelScan
    NB Defense

    Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com.

    Additional tools and resources to check out:
    Protect AI Guardian: Zero Trust for ML Models

    Recon: Automated Red Teaming for GenAI

    Protect AI’s ML Security-Focused Open Source Tools

    LLM Guard Open Source Security Toolkit for LLM Interactions

    Huntr - The World's First AI/Machine Learning Bug Bounty Platform

    36 min
  • Exploring AI/ML Security Risks: At Black Hat USA 2023 with Protect AI

    Send a text

    Watch the video for this episode at: https://mlsecops.com/podcast/exploring-ai/ml-security-risks-at-black-hat-usa-2023

    This episode of The MLSecOps Podcast features expert security leaders who sat down at Black Hat USA 2023 last week with team members from Protect AI to talk about various facets of AI and machine learning security:

    - What is the overall state of the AI/ML security realm at this time?
    - What is currently the largest threat to AI and machine learning security?
    - What is the most important thing we need to focus on at this time in the machine learning security space?
    - Plus much more!

    Guest Speaker Information:

    Adam Nygate
    Adam is the founder of huntr.dev (recently acquired by Protect AI), the place to protect open-source software, where they not only provide bug and fix bounties, but have built a platform that automates triaging, creating a noise-free environment for maintainers. Adam’s work with the huntr.dev community of over 10k security contributors enabled them to become the 5th largest CNA in the world in 2022. Adam was included in the OpenUK’s 2022 Honours List and has been featured in articles by AWS, Nominet and on podcasts such as Zero Hour and Absolute AppSec.

    Daniel Miessler
    "Daniel is the founder of Unsupervised Learning, a company focused on building products that help companies, organizations, and people identify, articulate, and execute on their purpose in the world. Daniel has over 20 years of experience in Cybersecurity, and has spent the last several years focused on applying AI to business and human problems."

    Adam Shostack
    Check out Adam's fantastic book, "Threats: What Every Engineer Should Learn From Star Wars!"
    Adam is a leading expert on threat modeling, a consultant, expert witness, and game designer.

    Christina Liaghati, PhD
    Dr. Liaghati is the AI Strategy Execution & Operations Manager, AI & Autonomy Innovation Center at MITRE, and a wealth of knowledge about all things MITRE ATLAS and the ML system attack chain. She is a highly sought-after speaker and is a repeat guest of ours on The MLSecOps Pod

    Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com.

    Additional tools and resources to check out:
    Protect AI Guardian: Zero Trust for ML Models

    Recon: Automated Red Teaming for GenAI

    Protect AI’s ML Security-Focused Open Source Tools

    LLM Guard Open Source Security Toolkit for LLM Interactions

    Huntr - The World's First AI/Machine Learning Bug Bounty Platform

    36 min
  • Everything You Need to Know About Hacker Summer Camp 2023

    Send a text

    Welcome back to The MLSecOps Podcast for this week's episode, “Everything You Need to Know About Hacker Summer Camp 2023.” 

    This week, our show is hosted by Protect AI's Chief Information Security Officer, Diana Kelley, and Diana talks with two more longtime security experts, Chloé Messdaghi and Dan McInerney, about all things related to what the security research community fondly refers to as Hacker Summer Camp. 

    The group discusses various events held throughout the course of this exciting week in Las Vegas, including what to expect at Black Hat [USA 2023] and DEF CON [31].

    (1:21) What is Hacker Summer Camp Week and what are the various events and Cons that take place during that time? 

    (3:58) It’s my first time attending Black Hat, DEF CON, Hacker Summer Camp Week, etc.: where can I find groups to attend with or to help me navigate where to go and what events to attend? 

    (9:53) Advice: if it’s my first time attending Black Hat, what other advice is there for me? What should I be thinking about? 

    (13:25) If I attend Black Hat, does that mean I’m automatically able to attend DEF CON or how does that work? (TL;DR separate passes are needed for each event)

    (14:14) Are certain personas more welcomed at specific conferences? 

    (15:49) What are some interesting panel talks we should know about? 

    (20:53) There are a couple of other conferences going on during “Summer Camp Week” - BSides Las Vegas, Squadcon, Diana Initiative. What are those? When are they taking place? Can I go to all of them? How does that work?

    (23:26) What AI/ML security trends are happening? What should I be looking for at Black Hat and DEF CON this year in terms of talks and research?

    (28:55) How can I determine if a particular talk is going to be worth my time? 

    (32:54) Any other tips on how to stay healthy and safe (both physical and electronic safety) throughout the week?

    Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com.

    Additional tools and resources to check out:
    Protect AI Guardian: Zero Trust for ML Models

    Recon: Automated Red Teaming for GenAI

    Protect AI’s ML Security-Focused Open Source Tools

    LLM Guard Open Source Security Toolkit for LLM Interactions

    Huntr - The World's First AI/Machine Learning Bug Bounty Platform

    39 min
  • Privacy Engineering: Safeguarding AI & ML Systems in a Data-Driven Era; With Guest Katharine Jarmul

    Send a text

    Welcome to The MLSecOps Podcast, where we dive deep into the world of machine learning security operations. In this episode, we talk with the renowned Katharine Jarmul. Katharine is a Principal Data Scientist at Thoughtworks, and the author of the popular new book, Practical Data Privacy. 

    Katharine also writes a blog titled, Probably Private, where she writes about data privacy, data security, and the intersection of data science and machine learning. 


    We cover a lot of ground in this conversation; from the more general data privacy and security risks associated with ML models, to more specific cases such as the case with OpenAI’s ChatGPT. We also touch on things like how GDPR and other regulatory frameworks put a spotlight on the privacy concerns we all have when it comes to the massive amount of data collected by models. Where does the data come from? How is it collected? Who gives consent? What if somebody wants to have their data removed?

    We also get into how organizations and professionals such as business leaders, data scientists, and ML practitioners can address these challenges when it comes to risks surrounding data, privacy, security, and reputation. We also explore the practices and processes that need to be implemented in order to integrate “Privacy by Design” into the machine learning lifecycle. 


    Katharine is a wealth of knowledge and insight into these data privacy issues. As always, thanks for listening to the podcast, for reading the transcript, and supporting the show in any way you can.


    With that, we hope you enjoy our conversation with Katharine Jarmul.


    Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com.

    Additional tools and resources to check out:
    Protect AI Guardian: Zero Trust for ML Models

    Recon: Automated Red Teaming for GenAI

    Protect AI’s ML Security-Focused Open Source Tools

    LLM Guard Open Source Security Toolkit for LLM Interactions

    Huntr - The World's First AI/Machine Learning Bug Bounty Platform

    47 min
  • The Intersection of MLSecOps and DataPrepOps; With Guest: Jennifer Prendki, PhD

    Send a text

    On this week’s episode from The MLSecOps Podcast, we have the pleasure of hearing from Dr. Jennifer Prendki, founder and CEO of Alectio - The DataPrepOps Company. Alectio’s name comes from a blend of the acronym “AL,” standing for Active Learning, and the Latin term for the word “selection,” which is “lectio.”

    In this episode, Dr. Prendki defines DataPrepOps for us and describes its contrasts to MLOps, along with how DataPrepOps intersects with MLSecOps best practices. She also discusses data quality, security risks in data science, and the role that data curation plays in helping to mitigate security risks in ML models.

    Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com.

    Additional tools and resources to check out:
    Protect AI Guardian: Zero Trust for ML Models

    Recon: Automated Red Teaming for GenAI

    Protect AI’s ML Security-Focused Open Source Tools

    LLM Guard Open Source Security Toolkit for LLM Interactions

    Huntr - The World's First AI/Machine Learning Bug Bounty Platform

    35 min
  • The Evolved Adversarial ML Landscape; With Guest: Apostol Vassilev, NIST

    Send a text

    In this episode, we explore the National Institute of Standards and Technology (NIST) white paper, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations. The report is co-authored by our guest for this conversation; Apostol Vassilev, NIST Research Team Supervisor. Apostol provides insights into the motivations behind this initiative and the collaborative research methodology employed by the NIST team.

    Apostol shares with us that this taxonomy and terminology report is part of the Trustworthy & Responsible AI Resource Center that NIST is developing. 

    Additional tools in the resource center include NIST’s AI Risk Management Framework (RMF), the OECD-NIST Catalogue of AI Tools and Metrics, and another crucial publication that Apostol co-authored called Towards a Standard for Identifying and Managing Bias in Artificial Intelligence.

    The conversation then focuses on the evolution of adversarial ML (AdvML) attacks, including prominent techniques like prompt injection attacks, as well as other emerging threats amidst the rise of large language model applications. Apostol discusses the changing AI and computing infrastructure and the scale of defenses required as a result of these changes. 

    Concluding the episode, Apostol shares thoughts on enhancing ML security practices and invites stakeholders to contribute to the ongoing development of the AdvML taxonomy and terminology white paper. 

    Join us now for a thought-provoking discussion that sheds light on NIST's efforts to further define the terminology of adversarial ML and develop a comprehensive taxonomy of concepts that will aid industry leaders in creating additional standards and guides.

    Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com.

    Additional tools and resources to check out:
    Protect AI Guardian: Zero Trust for ML Models

    Recon: Automated Red Teaming for GenAI

    Protect AI’s ML Security-Focused Open Source Tools

    LLM Guard Open Source Security Toolkit for LLM Interactions

    Huntr - The World's First AI/Machine Learning Bug Bounty Platform

    31 min
  • Navigating the Challenges of LLMs: Guardrails AI to the Rescue; With Guest: Shreya Rajpal

    Send a text

    In “Navigating the Challenges of LLMs: Guardrails to the Rescue,” Protect AI Co-Founders, Daryan Dehghanpisheh and Badar Ahmed, interview the creator of Guardrails AI, Shreya Rajpal.

    Guardrails AI is an open source package that allows users to add structure, type, and quality guarantees to the outputs of large language models (LLMs). 

    In this highly technical discussion, the group digs into Shreya’s inspiration for starting the Guardrails project, the challenges of building a deterministic “guardrail” system on top of probabilistic large language models, and the challenges in general (both technical and otherwise) that developers face when building applications for LLMs. 

    If you’re an engineer or developer in this space looking to integrate large language models into the applications you’re building, this episode is a must-listen and highlights important security considerations.


    Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com.

    Additional tools and resources to check out:
    Protect AI Guardian: Zero Trust for ML Models

    Recon: Automated Red Teaming for GenAI

    Protect AI’s ML Security-Focused Open Source Tools

    LLM Guard Open Source Security Toolkit for LLM Interactions

    Huntr - The World's First AI/Machine Learning Bug Bounty Platform

    40 min
  • Indirect Prompt Injections and Threat Modeling of LLM Applications; With Guest: Kai Greshake

    Send a text

    This talk makes it increasingly clear. The time for machine learning security operations - MLSecOps - is now. 

    In “Indirect Prompt Injections and Threat Modeling of LLM Applications,” (transcript here -> https://bit.ly/45DYMAG) we dive deep into the world of large language model (LLM) attacks and security. Our conversation with esteemed cyber security engineer and researcher, Kai Greshake, centers around the concept of indirect prompt injections, a novel adversarial attack and vulnerability in LLM-integrated applications, which Kai has explored extensively. 

    Our host, Daryan Dehghanpisheh, is joined by special guest-host (Red Team Director and prior show guest) Johann Rehberger to discuss Kai’s research, including the potential real-world implications of these security breaches. They also examine contrasts to traditional security injection vulnerabilities like SQL injections. 

    The group also discusses the role of LLM applications in everyday workflows and the increased security risks posed by their integration into various industry systems, including military applications. The discussion then shifts to potential mitigation strategies and the future of AI red teaming and ML security. 



    Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com.

    Additional tools and resources to check out:
    Protect AI Guardian: Zero Trust for ML Models

    Recon: Automated Red Teaming for GenAI

    Protect AI’s ML Security-Focused Open Source Tools

    LLM Guard Open Source Security Toolkit for LLM Interactions

    Huntr - The World's First AI/Machine Learning Bug Bounty Platform

    37 min
  • Responsible AI: Defining, Implementing, and Navigating the Future; With Guest: Diya Wynn

    Send a text

    In this episode of The MLSecOps Podcast, Diya Wynn, Sr. Practice Manager in Responsible AI in the Machine Learning Solutions Lab at Amazon Web Services shares her background and the motivations that led her to pursue a career in Responsible AI. 

    Diya shares her passion for work related to diversity, equity, and inclusion (DEI), and how Responsible AI offers a unique opportunity to merge her passion for DEI with what her core focus has always been: technology. She explores the definition of  Responsible AI as an operating approach focused on minimizing unintended impact and maximizing benefits. The group also spends some time in this episode discussing Generative AI and its potential to perpetuate biases and raise ethical concerns. 

    Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com.

    Additional tools and resources to check out:
    Protect AI Guardian: Zero Trust for ML Models

    Recon: Automated Red Teaming for GenAI

    Protect AI’s ML Security-Focused Open Source Tools

    LLM Guard Open Source Security Toolkit for LLM Interactions

    Huntr - The World's First AI/Machine Learning Bug Bounty Platform

    34 min

About The MLSecOps Podcast

From the publisher's feed

Welcome to The MLSecOps Podcast, presented by Protect AI. Here we explore the world of machine learning security operations, a.k.a., MLSecOps. From preventing attacks to navigating new AI…

More shows like The MLSecOps Podcast

Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

Super Data Science: ML & AI Podcast with Jon Krohn

301 Listeners

Practical AI by Daniel Whitenack and Chris Benson

Practical AI

204 Listeners

The AI Daily Brief: Artificial Intelligence News and Analysis by Nathaniel Whittemore

The AI Daily Brief: Artificial Intelligence News and Analysis

663 Listeners