Algorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processing

Navigating AI Governance and Compliance

Patrick Sullivan is Vice President of Strategy and Innovation at A-LIGN and an expert in cybersecurity and AI compliance with over 25 years of experience.

Patrick shares his career journey, discusses his passion for educating executives and directors on effective governance, and explains the critical role of management systems like ISO 42001 in AI compliance.

We discuss the complexities of AI governance, risk assessment, and the importance of clear organizational context.

Patrick also highlights the challenges and benefits of AI assurance and offers insights into the changing landscape of AI standards and regulations.

Chapter Markers

00:00 Introduction

00:23 Patrick's Career Journey

02:31 Focus on AI Governance

04:19 Importance of Education and Internal Training

08:08 Involvement in Industry Associations

14:13 AI Standards and Governance

20:06 Challenges with preparing for AI Certification

28:04 Future of AI Assurance

To subscribe to the weekly articles: https://riskinsights.com.au/blog#subscribe

About this podcast

A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI.

Hosted by Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).

Mitigating AI Risks

Ryan Carrier is founder and executive director of ForHumanity, a non-profit focused on mitigating the risks associated with AI, autonomous, and algorithmic systems.

With 25 years of experience in financial services, Ryan discusses ForHumanity's mission to analyze and mitigate the downside risks of AI to benefit society.

The conversation includes insights on the foundation of ForHumanity, the role of independent AI audits, educational programs offered by the ForHumanity AI Education and Training Center, AI governance, and the development of audit certification schemes.

Ryan also highlights the importance of AI literacy, stakeholder management, and the future of AI governance and compliance.

Chapter Markers

00:00 Introduction to Ryan Carrier and ForHumanity

00:57 Ryan's Background and Journey to AI

02:10 Founding ForHumanity: Mission and Early Challenges

05:15 Developing Independent Audits for AI

08:02 ForHumanity's Role and Activities

17:26 Education Programs and Certifications

29:21 AI Literacy and Future of Independent Audits

42:06 Getting Involved with ForHumanity

To subscribe to the weekly articles: https://riskinsights.com.au/blog#subscribe

About this podcast

A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI.

Hosted by Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).

Spoken (by a human) version of this article.

• Public AI audit reports aren't universally required; they mainly apply to high-risk applications and/or specific jurisdictions.
• The push for transparency primarily concerns independent audits, not internal reviews.
• Prepare by implementing ethical AI practices and conducting regular reviews.

Note: High-risk AI systems in banking and insurance are subject to specific requirements

Links

• AI and algorithm audit guidelines vary widely and are not universally applicable. We discussed this in a previous article, outlining how the appropriateness of audit guidance depends on your circumstances.
• Audit vs Review: we explored this topic in depth in a previous article. 

To subscribe to the weekly articles: https://riskinsights.com.au/blog#subscribe

About this podcast

A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI.

Hosted by Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).

Spoken by a human version of this article.

• Knowing the basics of substantive testing vs. controls testing can help you determine if the review will meet your needs.
• Substantive testing directly identifies errors or unfairness, while controls testing evaluates governance effectiveness. The results/conclusions are different.
• Understanding these differences can also help you anticipate the extent of your team's involvement during the review process.

Links
 This article details a (largely) substantive testing method for accuracy reviews.

To subscribe to the weekly articles: https://riskinsights.com.au/blog#subscribe

About this podcast

A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI.

Hosted by Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).

Spoken by a human version of this article.

Ongoing education helps everyone understand their role in responsibly developing and using algorithmic systems.

Regulators and standard-setting bodies emphasise the need for AI literacy across all organisational levels.

Links

• ForHumanity - join the growing community here. 
• ForHumanity - free courses here.
• IAIS: The International Association of Insurance Supervisors is developing a guidance paper on the supervision of AI.
• DNB: De Nederlandsche Bank - 6 general principles for the use of AI in the financial sector.
• ASIC: The Australian Securities & Investments Commission - report.
• NIST: The National Institute of Standards and Technology - AI Risk Management Framework.
• EU AI Act: The European Union Artificial Intelligence Act - specific expectation about “AI literacy”.

To subscribe to the weekly articles: https://riskinsights.com.au/blog#subscribe

About this podcast

A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI.

Hosted by Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).

Spoken by a human version of this article.

The terminology – “audit” vs “review” - is important, but clarity about deliverables is more important when commissioning algorithm integrity assessments.

Audits are formal, with an opinion or conclusion that can often be shared externally. Reviews come in various forms and typically produce recommendations, for internal use.

Regardless of the terminology you use, when commissioning an assessment, clearly define and document the expected deliverable, including the report content and intended distribution, to ensure expectations are met.

To subscribe to the weekly articles: https://riskinsights.com.au/blog#subscribe

About this podcast

A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI.

Hosted by Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).

Spoken (by a human) version of this article.

• Outcome-focused accuracy reviews directly verify results, offering more robust assurance than process-focused methods.
• This approach can catch translation errors, unintended consequences, and edge cases that process reviews might miss.
• While more time-consuming and complex, outcome-focused reviews provide deeper insights into system reliability and accuracy.

This article explains why verifying outcomes is preferred over tracing through processes, and how it works.

To subscribe to the weekly articles: https://riskinsights.com.au/blog#subscribe

About this podcast

A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI.

Hosted by Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).

Spoken (by a human) version of this article.

Documentation makes it easier to consistently maintain algorithm integrity.

This is well known.

But there are lots of types of documents to prepare, and often the first hurdle is just thinking about where to start.

So this simple guide is meant to help do exactly that – get going.

To subscribe to the weekly articles: https://riskinsights.com.au/blog#subscribe

About this podcast

A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI.

Hosted by Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).

Spoken (by a human) version of this article.

Banks and insurers are increasingly using external data; using them beyond their intended purpose can be risky (e.g. discriminatory).

Emerging regulations and regulatory guidance emphasise the need for active oversight by boards, senior management to ensure responsible use of external data.

Keeping the customer top of mind, asking the right questions, and focusing on the intended purpose of the data, can help reduce the risk.

Law and guideline mentioned in the article: 

• Colorado's External Consumer Data and Information Sources (ECDIS) law 
• New York's proposed circular letter.

To subscribe to the weekly articles: https://riskinsights.com.au/blog#subscribe

About this podcast

A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI.

Hosted by Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).

Spoken (by a human) version of this article.

Banks and insurers sometimes lose sight of their customer-centric purpose when assessing AI/algorithm risks, focusing instead on regular business risks and regulatory concerns.

Regulators are noticing this disconnect.

This article aims to outline why the disconnect happens and how we can fix it.

Report mentioned in the article: ASIC, REP 798 Beware the gap: Governance arrangements in the face of AI innovation.

To subscribe to the weekly articles: https://riskinsights.com.au/blog#subscribe

About this podcast

A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI.

Hosted by Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).