Value Driven Data Science

With all the reports about the spread of misinformation and disinformation on social media, sometimes it feels like one of the biggest threats to democracy is technology. But no technology is inherently good or bad. It’s how you use it that matters. And just as technology has the potential to harm democracy, it also has the potential to enhance it.

In this episode, Vikram Oberoi joins Dr Genevieve Hayes to discuss how he has been using generative AI and large language models (LLMs) to enhance people’s access to NYC council meetings through his work on citymeetings.nyc.

Guest Bio

Vikram Oberoi is a software engineer, fractional CTO and co-owner of Baxter HQ, a boutique early-stage tech product development firm. He also built and operates citymeetings.nyc, an LLM powered tool to make New York City council meetings accessible.

Highlights

• (00:00) Meet Vikram Oberoi
• (01:31) Overview of citymeetings.nyc
• (07:50) Vikram’s journey into local politics
• (12:05) Technical aspects of citymeetings.nyc
• (18:41) Dealing with AI hallucinations
• (25:00) Understanding the different types of AI errors
• (26:05) Case study: Honeycomb’s query feature
• (26:59) Reinforcement learning with human feedback
• (28:32) Choosing between Claude and GPT
• (31:42) The importance of context windows
• (40:31) Effective prompt engineering tips
• (46:11) Final advice for data scientists

Links

• citymeetings.nyc
• Vikram’s website
• Vikram’s talk at NYC School of Data about citymeetings.nyc
• Follow Vikram on X
• Connect with Genevieve on LinkedIn
• Be among the first to hear about the release of each new podcast episode by signing up HERE

Succeeding in stock market investing is all about timing – buying low, selling high and being able to read the signs to determine when things are going to change. But as anyone who’s ever tried to get rich through stock trading can tell you, this is easier said than done.

Given the massive amounts of financial data published each day, for people who aren’t experts in the field, it can be too hard to spot the patterns and keep up with the constant change. As a result, many people are either investing in markets based on guesswork or not investing at all.

This is where AI can help, because there’s nothing that AI does better than finding patterns in large volumes of data. AI has the potential to democratize access to investment insights.

In this episode, Andrew Einhorn joins Dr Genevieve Hayes to discuss how AI can help ordinary investors find better investment opportunities than they could ever manage on their own.

Guest Bio

Andrew Einhorn is the CEO and co-founder of Levelfields, an AI-driven fintech application that automates arduous investment research so investors can find opportunities faster and easier. Before moving into finance, Andrew started his career as an epidemiologist and helped build a pandemic monitoring system for Georgetown Hospital. He also previously co-founded tech company Synoptus, has consulted for NASA and served as an advisor to a $65 billion hedge fund.

Highlights

• (00:06) Meet Andrew Einhorn
• (02:54) Andrew’s journey from public health to data science
• (07:55) The birth of Levelfields
• (19:35) Event-driven investment insights explained
• (26:22) AI and data science behind Levelfields
• (36:36) User experience and customisation
• (41:03) Future developments and final advice

Links

• Levelfields website
• Connect with Genevieve on LinkedIn
• Be among the first to hear about the release of each new podcast episode by signing up HERE

As a data scientist, there’s nothing worse than devoting months of your time to building a data product that appears to meet your stakeholders’ every need, only to find it never gets used. It’s depressing, demotivating and can be devastating for your career.

But as the old saying goes, “You can lead a horse to water, but you can’t make it drink”. Or can you?

In this episode, Brian T O’Neill joins Dr Genevieve Hayes to discuss how you can apply the best techniques from software product management and UI/UX design to create ML and AI products your stakeholders will love.

Guest Bio

Brian T O’Neill is the Founder and Principal of Designing for Analytics, an independent data product UI/UX design consultancy that helps data leaders turn ML & analytics into usable, valuable data products. He also advises on product and UI/UX design for startup founders in MIT’s Sandbox Innovation Fund; hosts the podcast Experiencing Data; founded The Data Product Leadership Community and maintains a career as a professional percussionist performing in Boston and internationally.

Highlights

• Introducing Brian T. O’Neill (00:19)
• Brian’s journey from music to data product design (02:16)
• Understanding the real needs of stakeholders (06:45)
• The importance of user-centered design in data products (09:33)
• Gaining insights through direct user interaction (12:16)
• Focusing on business and user experience outcomes (17:48)
• Debunking the myths of self-serve analytics and dashboarding (22:46)
• Data platforms vs. data products (27:26)
• Defining a data product: the value exchange principle (29:08)
• Designing human-centered data products (32:56)
• The CED framework: conclusions, evidence, data (36:01)
• Final advice for data scientists (45:06)

Links

• Brian’s mailing list
• Designing for Analytics
• Data Product Leadership Community
• CED framework
• Connect with Genevieve on LinkedIn
• Be among the first to hear about the release of each new podcast episode by signing up HERE

Two years ago, no one could imagine the impact generative AI would have on our world, and most of us can’t even begin to imagine the impact the next generation of AI will have on our world two years from now. The only thing that is certain is uncertainty.

But that uncertainty brings with it great opportunities and choices. We can choose to sit back and let the future of AI play out in front of us or engage with this new technology and shape the future of AI and the world as we know it.

In this episode, Dr Eric Daimler joins Dr Genevieve Hayes to discuss his extraordinary work in shaping the future of AI and what that future might look like.

Guest Bio

Dr. Eric Daimler is the Chair, CEO and Co-Founder of Conexus AI and has previously co-founded five other companies in the technology space. He served under the Obama Administration as a Presidential Innovation Fellow for AI and Robotics in the Executive Office of President, as the sole authority driving the agenda for U.S. leadership in research, commercialization, and public adoption of AI & Robotics. He is also the author of the upcoming book The Future is Formal: The Roadmap for Using Technology to Solve Society’s Biggest Problems.

Highlights

• (00:00) Meet Dr. Eric Daimler
• (01:46) Eric’s role in the Obama Administration
• (06:32) Challenges in government data integration
• (10:31) The importance of technical expertise in policy
• (16:06) Founding Connexus AI
• (18:09) Understanding category theory
• (20:51) Applications of Conexus AI
• (27:16) The future of AI: safe and symbolic
• (38:35) Insights from Eric’s upcoming book
• (47:49) Advice for data scientists and final thoughts

Links

• Connect with Eric on LinkedIn
• Conexus AI website
• Connect with Genevieve on LinkedIn
• Be among the first to hear about the release of each new podcast episode by signing up HERE

Chances are, you’re reading this summary on a device you didn’t build yourself. Why would you? Tech companies can build you a far better device for a much lower cost than you could ever manage alone. As with many other cases in life, this is an example of where it is better to buy than to build.

Yet, in building a data team, many organisations assume the only solution is to build from within. And although this may be the right solution for some organisations, building a solution isn’t right for all.

In this episode, Collin Graves joins Dr Genevieve Hayes to discuss what a bought solution might look like in the data science space, and whether it is right for you.

Guest Bio

Collin Graves is the CEO of North Labs, a leading fractional cloud data analytics firm that helps growing companies become data-driven. Before founding North Labs, he served with distinction in NATO Special Operations during his tenure with the US Air Force. He is also the author of the upcoming Data Revolution: Leading with Analytics and Winning from Day One.

Highlights

• (01:43) Collin’s journey from the US Air Force to data science
•  (09:53) The birth of North Labs: a fractional data analytics firm
•  (12:02) Scaling a one-man operation to a thriving business
•  (13:58) The challenges of using data in the industrial and manufacturing sector
•  (28:41) The power of outsourcing data science
•  (34:09) The future of data teams and the role of in-house expertise
•  (41:44) Insights from Collin’s upcoming book
•  (46:17) Final thoughts and advice for data scientists

Links

• Connect with Collin on LinkedIn
• North Labs website
• Connect with Genevieve on LinkedIn
• Be among the first to hear about the release of each new podcast episode by signing up HERE

When ChatGPT was first released, there was talk it would lead to traditional search engines, like Google, soon becoming obsolete. That was until users discovered generative AI’s one major drawback – it makes stuff up.

Because of the stochastic nature of ChatGPT, it is never going to be possible to completely eliminate hallucinations. However, there are ways to work around this issue. One such way is through leveraging knowledge graphs and retrieval augmented generation (or RAG).

In this episode, Kirk Marple joins Dr Genevieve Hayes to discuss how knowledge graphs and RAG can be leveraged to improve the quality of generative AI.

Guest Bio

Kirk Marple is the CEO and Technical Founder of Graphlit, serverless, cloud-native platform that streamlines the development of AI apps by automating unstructured data workflows and leveraging retrieval augmented generation.

Highlights

• (00:19) Meet Kirk Marple
• (01:22) Leveraging knowledge graphs and RAG
• (06:08) Challenges with named entity extraction
• (09:16) Cost implications of LLMs
• (12:17) Deep dive into RAG
• (16:58) Vector search explained
• (20:49) Graph databases and RAG
• (38:58) Future of RAG and AI
• (43:08) Final thoughts

Links

• Connect with Kirk on LinkedIn
• Graphlit website
• Follow Graphlit on X
• Connect with Genevieve on LinkedIn
• Be among the first to hear about the release of each new podcast episode by signing up HERE

For many people, data science is synonymous with machine learning and many data science courses are little more than overviews of the most used machine learning algorithms and techniques.

Where the majority of data science courses fall short is they neglect to bridge the gap between data science theory and business reality, resulting in many data scientists who are technically strong but unable to create value from their work. However, this doesn’t necessarily have to be the case.

In this episode, Douglas Squirrel joins Dr Genevieve Hayes to discuss systems and techniques data scientists and their managers can use to make data science teams profitable.

Guest Bio

Douglas Squirrel has been coding for forty-five years and has led software teams for twenty-five. He uses the power of conversations to create insane profits in technology organisations of all sizes. His experience includes growing software teams as a CTO in startups; consulting on product improvement; and coaching a wide variety of leaders in improving their conversations, aligning to business goals, and creating productive conflict.

Highlights

• Douglas Squirrel’s journey: From CTO to profitability guru (00:00)
• Integrating data science with business goals (10:58)
• The surprising technological growth in Africa (17:38)
• Overcoming the Walled Garden: strategies for tech team success (19:14)
• The Lean Startup approach to data science (26:48)
• The importance of direct feedback in data science (32:50)
• Transforming data science with human empathy (33:39)
• Leveraging action science for effective communication (42:46)
• Elephant Carpaccio (47:41)
• Techniques for data scientists to create business value (51:22)
• Creating productive conflict for business innovation (53:43)
• Final thoughts and resources (01:00:28)

Links

• Douglas Squirrel’s website
• Squirrel Squadron
• Connect with Genevieve on LinkedIn
• Be among the first to hear about the release of each new podcast episode by signing up HERE

One of the big promises of data science is its ability to combine multiple disparate datasets to produce value-creating insights. But this is only possible if you can get all those disparate datasets together, in the one location, to begin with. The has led to the rise of the data engineer and the data orchestration platform.

In this episode, Sandy Ryza joins Dr Genevieve Hayes to discuss the impact of the data scientist on the creation of the next generation of data orchestration tools.

Guest Bio

Sandy Ryza is a data scientist turned data engineer who is currently the lead engineer on the Dagster project, an open-source data orchestration platform used in MLOps, data science, IOT and analytics. He is also the co-author of Advanced Analytics with Spark.

Highlights

• Welcome to Value Driven Data Science (00:00)
• Introducing Sandy Ryza and his journey from data scientist to data engineer (01:30)
• Navigating the challenges of creating consistent data definitions within teams (05:11)
• The birth and development of Dagster (11:32)
• Dagster: A tool designed for data scientists (20:54)
• Final thoughts and advice for data scientists (37:29)

Links

• Connect with Sandy on LinkedIn
• Follow Sandy on X
• Dagster
• Connect with Genevieve on LinkedIn
• Be among the first to hear about the release of each new podcast episode by signing up HERE

From BuzzFeed Quizzes to the national census, it’s impossible to get through life without encountering surveys. However, not all surveys are created equal. As with everything else in data science, garbage going in will inevitably lead to garbage coming out.

In this episode, Kyle Block joins Dr Genevieve Hayes to look at practical techniques for designing surveys to ensure they deliver value, as well as approaches to analysing survey results, to maximise that value.

Guest Bio

Kyle Block is Head of Research at Gradient, an analytics agency that combines advanced statistical and machine learning techniques to answer difficult marketing challenges. He holds a Masters in Spatial Analysis from the University of Pennsylvania and has spent his career helping managers use data to make important decisions.

Talking Points

• What good survey design looks like.
• Advice on how to design effective surveys.
• How list experiments can be used to uncover true opinions around sensitive topics.
• How data science techniques can be applied to survey data analysis to maximise its value.
• What the future might hold for survey data analysis.

Links

• Connect with Kyle on LinkedIn
• Gradient Website
• Connect with Genevieve on LinkedIn
• Be among the first to hear about the release of each new podcast episode by signing up HERE

Most people have come to accept that the price of living in a technological world, and its associated convenience, is some loss of data privacy. However, few realise just how much privacy they are giving up.

In this episode, Dr Katharine Kemp joins Dr Genevieve Hayes to discuss data privacy challenges for consumers and data scientists in the age of AI.

Guest Bio

Dr Katharine Kemp is an Associate Professor in UNSW’s Faculty of Law and Justice and Deputy Director of the Allens Hub for Technology, Law and Innovation. Her research focuses on competition, data privacy and consumer protection regulation, including their application to digital platforms.

Talking Points

• What types of data are companies collecting about their customers?
• How companies currently de-identify customer data to ensure consumer privacy is protected.
• The effectiveness of data de-identification methods at truly protecting the privacy of individuals.
• The state of current consumer data privacy laws and how they are likely to evolve.
• The impact of generative AI tools, such as ChatGPT, on consumer data privacy.

Links

• UNSW Allens Hub for Technology, Law and Innovation
• Katharine’s Research (SSRN Page)
• Consumer Policy Research Centre
• Singled Out Report
• Connect with Genevieve on LinkedIn
• Be among the first to hear about the release of each new podcast episode by signing up HERE