Data Science at Home

What data transformation library should I use? Pandas vs Dask vs Ray vs Modin vs Rapids (Ep. 112)


Listen Later

In this episode I speak about data transformation frameworks available for the data scientist who writes Python code.

The usual suspect is clearly Pandas, as the most widely used library and de-facto standard. However when data volumes increase and distributed algorithms are in place (according to a map-reduce paradigm of computation), Pandas no longer performs as expected. Other frameworks play a role in such context. 

In this episode I explain the frameworks that are the best equivalent to Pandas in bigdata contexts.

Don't forget to join our Discord channel and comment previous episodes or propose new ones.

 

This episode is supported by Amethix Technologies

Amethix works to create and maximize the impact of the world’s leading corporations, startups, and nonprofits, so they can create a better future for everyone they serve. Amethix is a consulting firm focused on data science, machine learning, and artificial intelligence.

 

References

Pandas a fast, powerful, flexible and easy to use open source data analysis and manipulation tool - https://pandas.pydata.org/

  • Modin - Scale your pandas workflows by changing one line of code - https://github.com/modin-project/modin

  • Dask advanced parallelism for analytics https://dask.org/

  • Ray is a fast and simple framework for building and running distributed applications https://github.com/ray-project/ray

  • RAPIDS - GPU data science https://rapids.ai/

    ...more
    View all episodesView all episodes
    Download on the App Store

    Data Science at HomeBy Francesco Gadaleta

    • 4.2
    • 4.2
    • 4.2
    • 4.2
    • 4.2

    4.2

    71 ratings


    More shows like Data Science at Home

    View all
    This Week in Startups by Jason Calacanis

    This Week in Startups

    1,287 Listeners

    The McKinsey Podcast by McKinsey & Company

    The McKinsey Podcast

    395 Listeners

    Data Skeptic by Kyle Polich

    Data Skeptic

    479 Listeners

    Software Engineering Daily by Software Engineering Daily

    Software Engineering Daily

    624 Listeners

    HBR IdeaCast by Harvard Business Review

    HBR IdeaCast

    1,839 Listeners

    Soft Skills Engineering by Jamison Dance and Dave Smith

    Soft Skills Engineering

    288 Listeners

    Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

    Super Data Science: ML & AI Podcast with Jon Krohn

    303 Listeners

    NVIDIA AI Podcast by NVIDIA

    NVIDIA AI Podcast

    335 Listeners

    Data Engineering Podcast by Tobias Macey

    Data Engineering Podcast

    145 Listeners

    DataFramed by DataCamp

    DataFramed

    268 Listeners

    Practical AI by Practical AI LLC

    Practical AI

    209 Listeners

    Machine Learning Street Talk (MLST) by Machine Learning Street Talk (MLST)

    Machine Learning Street Talk (MLST)

    93 Listeners

    Latent Space: The AI Engineer Podcast by swyx + Alessio

    Latent Space: The AI Engineer Podcast

    95 Listeners

    This Day in AI Podcast by Michael Sharkey, Chris Sharkey

    This Day in AI Podcast

    209 Listeners

    The AI Daily Brief: Artificial Intelligence News and Analysis by Nathaniel Whittemore

    The AI Daily Brief: Artificial Intelligence News and Analysis

    594 Listeners