In this episode, Caterina Constantinescu dives deep into Large Language Models (LLMs), spotlighting top leaderboards, evaluation benchmarks, and real-world user perceptions. Plus, discover the challenges of dataset contamination and the intricacies of platforms like HELM and Chatbot Arena.

Additional materials: www.superdatascience.com/706

Interested in sponsoring a SuperDataScience Podcast episode? Visit JonKrohn.com/podcast for sponsorship information.

706: Large Language Model Leaderboards and Benchmarks

The latest machine learning, A.I., and data career topics from across both academia and industry are brought to you by host Dr. Jon Krohn on the Super Data Science Podcast.

As the quantity of data on our planet doubles every couple of years and with this trend set to continue for decades to come, there's an unprecedented opportunity for you to make a meaningful impact in your lifetime. In conversation with the biggest names in the data science industry, Jon cuts through hype to fuel that professional impact.

Whether you're curious about getting started in a data career or you're a deep technical expert, whether you'd like to understand what A.I. is or you'd like to integrate more data-driven processes into your business, we have inspiring guests and lighthearted conversation for you to enjoy.

We cover tools, techniques, and implementation tricks across data collection, databases, analytics, predictive modeling, visualization, software engineering, real-world applications, commercialization, and entrepreneurship − everything you need to crush it with data science.

Technology

Tech News

Podcasting

Gadgets

Software How-To

Hear Jon Krohn’s favorite five clips from his April interviews. Chief Scientist at Posit PBC Hadley Wickham on the subtle differences between Python and R. Professor of Business Analytics Barrett Thomas walks through the variables that companies should consider when using drones or any other tech to improve their business operations and bottom line. Aleksa Gordić, Founder of Runa AI believes an overhaul of the current educational system is long overdue. Bernard Marr discusses the future of GenAI and its impact on the world of work. And SuperDataScience founder Kirill Eremenko gives a lively workshop on gradient boosting.


Additional materials: www.superdatascience.com/782

Interested in sponsoring a SuperDataScience Podcast episode? Visit passionfroot.me/superdatascience for sponsorship information.

782: In Case You Missed It in April 2024

Sol Rashidi, a distinguished data executive who has served in C-suite roles at Fortune 100 companies, joins Jon Krohn to delve into successful enterprise AI strategies and the reasons behind the high turnover among Chief Data Officers. This episode provides an in-depth look at selecting AI projects that succeed and understanding the strategic value of patents in various industries. Benefit from Sol’s extensive experience and practical advice on navigating complex corporate challenges.

This episode is brought to you by AWS Inferentia (https://go.aws/3zWS0au) and AWS Trainium (https://go.aws/3ycV6K0). Interested in sponsoring a SuperDataScience Podcast episode? Visit passionfroot.me/superdatascience for sponsorship information.

In this episode you will learn:
• How CDOs and related roles have such high turnover because [09:40]
• The importance of building relationships in AI projects [17:01]
• How Sol's book "The AI Survival Guide" came about [20:44]
• How high-criticality, low-complexity AI projects are the ones with the highest probability of success [27:11]
• How Enterprise data security issues can be resolved with technologies like Protopia’s stained-glass data-masking solution [36:10]
• Why having great data engineers is essential [47:57]
• The value of patents [51:45]

Additional materials: www.superdatascience.com/781

781: Ensuring Successful Enterprise AI Deployments, with Sol Rashidi

Want to become a data scientist? Jon and Adam discuss the key steps to becoming a data scientist, with a focus on developing portfolio projects. Hear about the 10 project ideas Adam recommends in his book to help you stand out in the data science community.

Additional materials: www.superdatascience.com/780

Interested in sponsoring a SuperDataScience Podcast episode? Visit passionfroot.me/superdatascience for sponsorship information.

780: How to Become a Data Scientist, with Dr. Adam Ross Nelson

Tidyverse, ggplot2, and the secret to a tech company’s longevity: Hadley Wickham talks to Jon Krohn about Posit’s rebrand, Tidyverse and why it needs to be in every data scientist’s toolkit, and why getting your hands dirty with open-source projects can be so lucrative for your career.

This episode is brought to you by Intel and HPE Ezmeral Software (https://bit.ly/hpeintel). Interested in sponsoring a SuperDataScience Podcast episode? Visit passionfroot.me/superdatascience for sponsorship information.

In this episode you will learn:
• All about the Tidyverse [04:46]
• Hadley’s favorite R libraries [17:10]
• The goal of Posit [30:29]
• On bringing multiple programming languages together [36:02]
• The principles for a long-lasting tech company [52:10]
• How Hadley developed ggplot2 [55:24]
• How to contribute to the open-source community [1:05:43]

Additional materials: www.superdatascience.com/779

779: The Tidyverse of Essential R Libraries and their Python Analogues, with Dr. Hadley Wickham

Mixtral 8x22B is the focus on this week's Five-Minute Friday. Jon Krohn examines how this model from French AI startup Mistral leverages its mixture-of-experts architecture to redefine efficiency and specialization in AI-powered tasks. Tune in to learn about its performance benchmarks and the transformative potential of its open-source license.

Additional materials: www.superdatascience.com/778

Interested in sponsoring a SuperDataScience Podcast episode? Visit passionfroot.me/superdatascience for sponsorship information.

706: Large Language Model Leaderboards and Benchmarks

Download our free app to listen on your phone