The AI Fundamentalists

It's the end of 2023 and our first season. The hosts reflect on what's happened with the fundamentals of AI regulation, data privacy, and ethics. Spoiler alert: a lot! And we're excited to share our outlook for AI in 2024.

• AI regulation and its impact in 2024.
  • Hosts reflect on AI regulation discussions from their first 10 episodes, discussing what went well and what didn't.
  • Its potential impact on innovation. 2:36
  • AI innovation, regulation, and best practices. 7:05
• AI, privacy, and data security in healthcare. 11:08
  • Data scientists face contradictory goals of fairness and privacy in AI, with increased interest in balancing both in 2024.
  • HIPAA privacy and the increasing use of machine learning and AI in healthcare.
  • Special thanks to the team at Shifting Privacy Left for a more in-depth discussion about privacy and synthetic data.
• AI safety and ethics in NLP research. 15:40
  • Does OpenAI's closed model set a bad precedent for research?
  • The tension in NLP research: AI safety, and OpenAI's approach 
• Modeling mindset and reality between scientists and AI experts. 18:44
  • Thanks to special guests who joined us in 2023
  • Josh Pyle and his insights on bias and actuarial bias, and how they've applied to their work. See episode 10.
  • Christoph Molnar for explaining the differences in mindset between scientists and AI scientists when modeling reality.  See episode 4.
• AI ethics and its challenges in the industry. 21:46
  • Andrew Clark emphasizes the importance of understanding the basics of time-series analysis and choosing the right tools for the job, rather than relying on automated methods or blindly using existing techniques.
  • Sid Mangalik: AI ethics discussion needs to level up, as people are confusing models with AGI and not addressing practical issues.
  • Andrew Clark: AI ethics is at a bad crossroads, with high-level discussions divorced from reality and companies paying lip service to ethical concerns.
• AI ethics and responsible development. 26:10
  • Companies with ethical bodies and practices are better equipped to handle AI ethics concerns.
  • Andrew expresses concern about the ethical implications of AI, particularly in the context of Google's Gemini project.
  • Comparing AI safety to carbon credits and the importance of a proactive approach to ethical considerations.
• AI ethics and its importance in business. 29:31
  • Susan highlights the importance of ethics in AI, citing examples of challenges between board and executive teams.

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

Joshua Pyle joins us in a discussion about managing bias in the actuarial sciences. Together with Andrew's and Sid's perspectives from  both the economic and data science fields, they deliver an interdisciplinary conversation about bias that you'll only find here.

• OpenAI news plus new developments in language models. 0:03
  • The hosts get to discuss the aftermath of OpenAI and Sam Altman's return as CEO
  • Tension between OpenAI's board and researchers on the push for slow, responsible AI development vs fast, breakthrough model-making.
  • Microsoft researchers find that smaller, high-quality data sets can be more effective for training language models than larger, lower-quality sets (Orca 2).
  • Google announces Gemini, a trio of models with varying parameters, including an ultra-light version for phones 
• Bias in actuarial sciences with Joshua Pyle, FCAS. 9:29
  • Josh shares insights on managing bias in Actuarial Sciences, drawing on his 20 years of experience in the field.
  • Bias in actuarial work defined as differential treatment leading to unfavorable outcomes, with protected classes including race, religion, and more.
• Actuarial bias and model validation in ratemaking. 15:48
  • The importance of analyzing the impact of pricing changes on protected classes, and the potential for unintended consequences when using proxies in actuarial ratemaking.
  • Three major causes of unfair bias in ratemaking (Contingencies, Nov 2023)
  • Gaps in the actuarial process that could lead to bias, including a lack of a standardized governance framework for model validation and calibration.
• Actuarial standards, bias, and credibility. 20:45
  • Complex state-level regulations and limited data pose challenges for predictive modeling in insurance.
  • Actuaries debate definition and mitigation of bias in continuing education.
• Bias analysis in actuarial modeling. 27:16
  • The importance of identifying dislocation analysis in bias analysis.
  • Analyze two versions of a model to compare predictive power of including vs. excluding protected class (race).
• Bias in AI models in actuarial field. 33:56
  • Actuaries can learn from data scientists' tendency to over-engineer models.
  • Actuaries may feel excluded from the Big Data era due to their need to explain their methods
  • Standardization is needed to help actuaries identify and mitigate bias.
• Interdisciplinary approaches to AI modeling and governance. 42:11
  • Sid hopes to see more systematic and published approaches to addressing bias in the data science field.
  • Andrew emphasizes the importance of interdisciplinary collaboration between actuaries, data scientists, and economists to create more accurate and fair modeling systems.
  • Josh agrees and highlights the need for better governance structures to support this collaboration, citing the lack of good journals and academic silos as a challenge.

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

Episode 9. Continuing our series run about model validation. In this episode, the hosts focus on aspects of performance, why we need to do statistics correctly, and not use metrics without understanding how they work, to ensure that models are evaluated in a meaningful way.

• AI regulations, red team testing, and physics-based modeling. 0:03
  • The hosts discuss the Biden administration's executive order on AI and its implications for model validation and performance.
• Evaluating machine learning models using accuracy, recall, and precision. 6:52
  • The four types of results in classification: true positive, false positive, true negative, and false negative.
  • The three standard metrics are composed of these elements: accuracy, recall, and precision.
• Accuracy metrics for classification models. 12:36
  • Precision and recall are interrelated aspects of accuracy in machine learning.
  • Using F1 score and F beta score in classification models, particularly when dealing with imbalanced data.
• Performance metrics for regression tasks. 17:08
  • Handling imbalanced outcomes in machine learning, particularly in regression tasks.
  • The different metrics used to evaluate regression models, including mean squared error.
• Performance metrics for machine learning models. 19:56
  • Mean squared error (MSE) as a metric for evaluating the accuracy of machine learning models, using the example of predicting house prices.
  • Mean absolute error (MAE) as an alternative metric, which penalizes large errors less heavily and is more straightforward to compute.
• Graph theory and operations research applications. 25:48
  • Graph theory in machine learning, including the shortest path problem and clustering. Euclidean distance is a popular benchmark for measuring distances between data points. 
• Machine learning metrics and evaluation methods. 33:06
• Model validation using statistics and information theory. 37:08
  • Entropy, its roots in classical mechanics and thermodynamics, and its application in information theory, particularly Shannon entropy calculation. 
  • The importance the use case and validation metrics for machine learning models.

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

Episode 8. This is the first in a series of episodes dedicated to model validation. Today, we focus on model robustness and resilience. From complex financial systems to why your gym might be overcrowded at New Year's, you've been directly affected by these aspects of model validation.

AI hype and consumer trust (0:03) 

• FTC article highlights consumer concerns about AI's impact on lives and businesses (Oct 3, FTC)
• Increased public awareness of AI and the masses of data needed to train it led to increased awareness of potential implications for misuse.
• Need for transparency and trust in AI's development and deployment.

Model validation and its importance in AI development (3:42)

• Importance of model validation in AI development, ensuring models are doing what they're supposed to do.
• FTC's heightened awareness of responsibility and the need for fair and unbiased AI practices.
• Model validation (targeted, specific) vs model evaluation (general, open-ended).

Model validation and resilience in machine learning (8:26)

• Collaboration between engineers and businesses to validate models for resilience and robustness.
• Resilience: how well a model handles adverse data scenarios.
• Robustness: model's ability to generalize to unforeseen data.
• Aerospace Engineering: models must be resilient and robust to perform well in real-world environments.

Statistical evaluation and modeling in machine learning (14:09)

• Statistical evaluation involves modeling distribution without knowing everything, using methods like Monte Carlo sampling.
• Monte Carlo simulations originated in physics for assessing risk and uncertainty in decision-making.

Monte Carlo methods for analyzing model robustness and resilience (17:24)

• Monte Carlo simulations allow exploration of potential input spaces and estimation of underlying distribution.
• Useful when analytical solutions are unavailable.
• Sensitivity analysis and uncertainty analysis as major flavors of analyses.

Monte Carlo techniques and model validation (21:31)

• Versatility of Monte Carlo simulations in various fields.
• Using Monte Carlo experiments to explore semantic space vectors of language models like GPT.
• Importance of validating machine learning models through negative scenario analysis.

Stress testing and resiliency in finance and engineering (25:48)

• Importance of stress testing in finance, combining traditional methods with Monte Carlo techniques.
• Synthetic data's potential in modeling critical systems.
• Identifying potential gaps and vulnerabilities in critical systems.

Using operations research and model validation in AI development (30:13)

• Operations research can help find an equilibrium in overcrowding in gyms.
• Robust methods for solving complex problems in logistics and healthcare.
• Model validation's importance in addressing issues of bias and fairness in AI systems.

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

Episode 7.  To use or not to use? That is the question about digital twins that the fundamentalists explore. Many solutions continue to be proposed for making AI systems safer, but can digital twins really deliver for AI what we know they can do for physical systems? Tune in and find out.

Show notes

• Digital twins by definition. 0:03
  • Digital twins are one-to-one digital models of real-life products, systems, or processes, used for simulations, testing, monitoring, maintenance, or practice decommissioning.
  • The digital twin should be indistinguishable from the physical twin, allowing for safe and efficient problem-solving in a computerized environment.
• Digital twins in manufacturing and aerospace engineering. 2:22
  • Digital twins are virtual replicas of physical processes, useful in manufacturing and space, but often misunderstood as just simulations or models.
  • Sid highlights the importance of identifying digital twin trends and distinguishing them from simulations or sandbox environments.
  • Andrew emphasizes the need for data standards and ETL processes to handle different vendors and data forms, clarifying that digital twins are not a one-size-fits-all solution.
• Digital twins, AI models, and validation in a hybrid environment. 6:51
  • Validation is crucial for deploying mission-critical AI models, including generative AI.
  • Sid clarifies the misconception that AI models can directly replicate physical systems, emphasizing the importance of modeling specific data and context.
  • Andrew and Susan discuss the confusion around modeling and its limitations, including the need to validate models on specific datasets and avoid generalizing across contexts.
  • Referenced article from Venture Beat, 10 digital twin trends for 2023
• Digital twins, IoT, and their applications. 11:05
  • Susan and Sid discuss the limitations of digital twins, including their inability to interact with the real world and the complexity of modeling systems.
  • They reference a 2012 NASA paper that popularized the term "digital twin" and highlight the potential for confusion in its application to various industries.
  • Sid: Digital twinning requires more than just IoT devices, it's a complex process that involves monitoring and IoT devices across the physical system to create a perfect digital twin.
  • Andrew: Digital twins raise security and privacy concerns, especially in healthcare, where there are lots of IoT devices and personal data that need to be protected.
• Data privacy and security in digital twin technology. 17:03
  • Digital twins and data privacy face off in IoT debate.
  • Susan and Andrew discuss data privacy concerns with AI and IoT, highlighting the potential for data breaches and lack of transparency.
• Digital twins in healthcare and technology. 20:16
  • Susan and Andrew discuss digital twins in various industries, emphasizing their importance and technical complexities.
  • Digital twin technology has a higher barrier to entry due to data security and privacy concerns, requiring intentional decision-making and long-term planning.

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

Episode 6. What does systems engineering have to do with AI fundamentals? In this episode, the team discusses what data and computer science as professions can learn from systems engineering, and how the methods and mindset of the latter can boost the quality of AI-based innovations.

Show notes

•  News and episode commentary 0:03
  • ChatGPT usage is down for the second straight month.
  • The importance of understanding the data and how it affects the quality of synthetic data for non-tabular use cases like text. (Episode 5, Synthetic data)
  • Business decisions. The 2012 case of Target using algorithms in their advertising. (CIO,  June 2023)
• Systems engineering thinking. 3:45
  • The difference between building algorithms and building models, and building systems. 
  • The term systems engineering came from Bell Labs in the 1940s, and came into its own with the NASA Apollo program.
  • A system is a way of looking at the world. There's emergent behavior, complex interactions and relationships between data.
  • AI systems and ML systems are often distant from the expertise of people who do systems engineering.
• Learning the hard way. 9:25
  • Systems engineering is about doing things the hard way, learning the physical sciences, math and how things work.
  • What else can be learned from the Apollo program.
  • Developing a system, and how important it is to align the importance of criticality and safety of the project.
  • Systems engineering is often associated incorrectly with waterfall in software engineering, 
• What is a safer model to build? 14:26
  • What is a safer model, and how is systems engineering going to fit in with this world?
  • The data science hacker culture can be counterintuitive to this approach 
  • For example, actuaries have a professional code of ethics and a set way that they learn.
• Step back and review your model. 18:26
  • Peer review your model and see if they can break it and stress-test it. Build monitoring around knowing where the fault points are and also talk to business leaders.
  • Be careful about the other impacts that can have on the business or externally on the people who start using it.
  • Marketing this type of engineering as robustness of the model, identifying what it is good at and what it's bad at, and that in itself can be a piece of selling.
  • Systems thinking gives a chance to create lasting models and lasting systems, not just models.
• How can you think of modeling as a system? 23:23
  • Andrew shares his thoughts on the importance of thinking holistically about the problem and creating a consistent, consistent, and reliable model.
  • Traceability and understanding of the system is the secret weapon. Understanding what tools from the box were used at which time, and the impact that it will have on either your customers or the decisions that your business makes on behalf of your customers.

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

Episode 5. This episode about synthetic data is very real. The fundamentalists uncover the pros and cons of synthetic data; as well as reliable use cases and the best techniques for safe and effective use in AI. When even SAG-AFTRA and OpenAI make synthetic data a household word, you know this is an episode you can't miss.

Show notes

• What is synthetic data? 0:03
  • Definition is not a succinct one-liner, which is one of the key issues with assessing synthetic data generation.
  • Using general information scraped from the web for ML is backfiring.
• Synthetic data generation and data recycling. 3:48
  • OpenAI is running against the problem that they don't have enough data and the scale at which they're trying to operate.
  • The poisoning effect that happens when trying to take your own data.
  • Synthetic data generation is not a panacea. It is not an exact science. It's more of an art than a science.
• The pros and cons of using synthetic data. 6:46
  • The pros and cons of using synthetic data to train AI models, and how it differs from traditional medical data.
  • The importance of diversity in the training of AI models.
  • Synthetic data is a nuanced field, taking away the complexity of building data that is representative of a solution.
• Differences between randomized and synthetic data. 9:52
  • Differential privacy is a lot more difficult to execute than a lot of people are talking about.
  • Anonymization is a huge piece of the application for the fairness bias, especially with larger deployments.
  • The hardest part is capturing complex interrelationships. (i.e. Fukushima reactor testing wasn't high enough)
• The pros and cons of ChatGPT. 13:54
  • Invalid use cases for synthetic data in more depth,
  • Examples where humans cannot anonymize effectively
  • Creating new data for where the company is right now before diving into the use cases; i.e. differential privacy.
• Mentally meaningful use cases for synthetic data. 16:38
  • Meaningful use cases for synthetic data, using the power of synthetic data correctly to generate outcomes that are important to you.
  • Pros and cons of using synthetic data in controlled environments.
• The fallacy of "fairness through awareness". 18:39
  • Synthetic data is helpful for stress testing systems, edge case scenario thought experiments, simulation, stress testing system design, and scenario-based methodologies.
  • The recent push to use synthetic data.
• Data augmentation and digital twin work. 21:26
  •  Synthetic data as the only data is where the difficulties arise.
  • Data augmentation is a better use case for synthetic data.
  • Examples of digital twin methodology to create a virtual twin of a physical system.
  • How to get synthetic data through intelligently sampling the original dataset.
• The importance of knowing the history of data. 27:16
  • Need to re-familiarize ourselves with these techniques in the context of the financial crisis
  • One of the key areas where synthetic data can be very powerful is when looking at past tabular data and the difference between use cases.

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

Episode 4. The AI Fundamentalists welcome Christoph Molnar to discuss the characteristics of a modeling mindset in a rapidly innovating world. He is the author of multiple data science books including Modeling Mindsets, Interpretable Machine Learning, and his latest book Introduction to Conformal Prediction with Python. We hope you enjoy this enlightening discussion from a model builder's point of view.

To keep in touch with Christoph's work, subscribe to his newsletter Mindful Modeler - "Better machine learning by thinking like a statistician. About model interpretation, paying attention to data, and always staying critical."

Summary

• Introduction. 0:03
  • Introduction to the AI fundamentalists podcast.
  • Welcome, Christopher Molnar
• What is machine learning? How do you look at it? 1:03
  • AI systems and machine learning systems.
  • Separating machine learning from classical statistical modeling.
• What’s the best machine learning approach? 3:41
  • Confusion in the space between statistical learning and machine learning.
  • The importance of modeling mindsets.
  • Different approaches to using interpretability in machine learning.
  • Holistic AI in systems engineering.
• Modeling is the most fun part but also the beginning. 8:19
  • Modeling is the most fun part of machine learning.
  • How to get lost in modeling.
• How can we use the techniques in interpretable ML to create a system that we can explain to stakeholders that are non-technical? 10:36
  • How to interpret at the non-technical level.
  • Reproducibility is a big part of explainability.
• Conformal prediction vs. interpretability tools. 12:51
  • Explanability to a data scientist vs. a regulator.
  • Interoperability is not a panacea.
  • Conformal prediction with Python.
  • Roadblocks to conformal prediction being used in the industry.
• What’s the best technique for a job in data science? 17:20
  • The bandwagon effect of Netflix and machine learning.
  • The mindset difference between data science and other professions.
• Machine learning is always catching up with the best practices in the industry. 19:21
  • The machine learning industry is catching up with best practices.
  • Synthetic data to fill in gaps.
  • The barrier to entry in machine learning.
  • How to learn from new models.
• How to train your mindset before you start modeling. 23:52
  • The importance of simplifying two different mindsets.
  • Introduction to conformal prediction with Python.

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

Episode 3.  Get ready because we're bringing stats back! An AI model can only learn from the data it has seen. And business problems can’t be solved without the right data. The Fundamentalists break down the basics of data from collection to regulation to bias to quality in AI. 

• Introduction to this episode
  • Why data matters.
• How do big tech's LLM models stack up to the proposed EU AI Act?
  • How major models such as Open AI and Bard stack up against current regulations.
  • Stanford HAI - Do Foundation Model Providers Comply with the Draft EU AI Act?
  • Risk management documentation and risk management.
• The EU is adding teeth outside of the banking and financial sectors now.
  • Time - Exclusive: OpenAI Lobbied the E.U. to Water Down AI Regulation
• Bringing stats back: Why does data matter in all this madness?
  • How AI is taking us away from human intelligence.
  • Having quality data and bringing stats back!
  • The importance of having representative data, sampling data
• What are your business objectives? Don’t just throw data into it.
  • Understanding the use case of the data.
  • GDPR and EU AI regulations.
  • AI field caught off guard by new regulations.
  • Expectations for regulatory data.
• What is data governance? How do you validate data?
  • Data management, data governance, and data quality.
  • Structured data collection for financial companies.
• What else should we learn about our data collection and data processes?
  • Example: US Census data collection and data processes.
  • The importance of representativeness and being representative of the community in the census.
  • Step one, the fine curation of data, the intentional and knowledgeable creation of data that meets the specific business need.
  • Step two, fairness through awareness.
• The importance of data curation and data selection in data quality.
  • What data quality looks like at a high level.
  • Rights to be forgotten.
• The importance of data provenance and data governance in data science.
  • Synthetic data and privacy.
• Data governance seems to be 40 % of the path to AI model governance. What else needs to be in place?
  • What companies are missing with machine learning.
  • The impact that data will have on the future of AI.
  • The future of general AI in the future.

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

Show Notes

• What’s NOT new and what is new in the world of LLMs. 3:10
  •  Getting back to the basics of modeling best practices and rigor.
• What is AI and subsequently LLM regulation going to look like for tech organizations? 5:55
  • Recommendations for reading on the topic.
  • Andrew talks about regulation, monitoring, assurance, and alarm.
• What does it mean to regulate generative AI models? 7:51
  • Concerns with regulating generative AI models.
  • Concerns about the call for regulation from Open AI.
• What is data privacy going to look like in the future? 10:16
  • Regulation of AI models and data privacy.
  • The NIST AI Risk Management Framework.
  • Making sure it's being used as a productivity tool.
  • How it's different from existing processes.
• What’s different about these models vs old models? 15:07
  • Public perception of new machine learning models vs old models.
  • Hallucination in the field.
• Does the use of chatbots change the tendency toward hallucinations? 17:27
  • Bing still suffers from the same problem with their LLMs.
  • Multi-objective modeling and multi-language modeling.
• What does truth-based AI look like? 20:17
  • Public perception vs. modeling best practices
  • Knowledge graphs vs. generative AI: ideal use cases for each
• Algorithms have a really interesting potential application which is a plugin library model. 23:00
  • Algorithms have an interesting potential application.
  • The benefits of a plugin library model.
• What’s the future of large language models? 25:35
  • Practical uses for ML and knowledge base knowledge databases.
  • Predictions on ML and ML-based databases.
  • Finding a way to make LLM useful.
  • Next episodes of the podcast.

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.