The Real Python Podcast

Becoming More Effective at Manipulating Data With Pandas

Listen Later

Do you wonder if you’re taking the right approach when shaping data in pandas? Is your Jupyter workflow getting out of hand? This week on the show, Matt Harrison talks about his new book, “Effective Pandas: Patterns for Data Manipulation.”

Matt discusses working as a corporate consultant and migrating Excel users toward Python. We explore several “NumPy-isms” that beginners get stuck on. Matt shares advice about chaining operations in pandas, which some developers find controversial.

Course Spotlight: Sorting Data in Python With Pandas

In this video course, you’ll learn how to sort data in a pandas DataFrame using the pandas sort functions sort_values() and sort_index(). You’ll learn how to sort by one or more columns and by index in ascending or descending order.

Topics:

  • 00:00:00 – Introduction
  • 00:01:32 – Working as a consultant
  • 00:03:39 – Moving from Excel to Python
  • 00:06:50 – Who is the book for?
  • 00:10:15 – Using real data for examples
  • 00:16:16 – Sponsor: CData Software
  • 00:16:58 – What are patterns for data manipulation?
  • 00:18:38 – Cleaning and preparing data
  • 00:21:33 – What concepts were you most eager to share?
  • 00:26:57 – An example of chaining operations in pandas
  • 00:33:20 – NumPy-isms and other challenges in learning pandas
  • 00:40:20 – The use of exercises throughout the book
  • 00:43:50 – Video Course Spotlight
  • 00:45:01 – Challenges of using color throughout the book
  • 00:51:40 – Avoiding the slow path in pandas
  • 00:56:03 – What are you excited about in the world of Python?
  • 00:56:58 – What would you like to learn next?
  • 00:58:16 – Effective Pandas book
  • 00:58:38 – Social connections
  • 00:58:58 – Thanks and goodbye

    • Show Links:

    • Effective Pandas Digital Book Discount Link
    • Matt Harrison’s Site - MetaSnake
    • pandas - Python Data Analysis Library
    • Law of Demeter - Wikipedia
    • PyCon 2022 - Welcome to PyCon US
    • Production-ready Docker packaging for Python developers | Python=>Speed

      • Level up your Python skills with our expert-led courses:

      • Using pandas to Make a Gradebook in Python
      • The pandas DataFrame: Working With Data Efficiently
      • Sorting Data in Python With pandas

        • Support the podcast & join our community of Pythonistas

        ...more
        View all episodesView all episodes
        Download on the App Store
        The Real Python PodcastBy Real Python
        • 4.7
        • 4.7
        • 4.7
        • 4.7
        • 4.7

        4.7

        139 ratings

        More shows like The Real Python Podcast

        View all
        The Changelog: Software Development, Open Source by Changelog Media

        The Changelog: Software Development, Open Source

        288 Listeners

        Software Engineering Daily by Software Engineering Daily

        Software Engineering Daily

        625 Listeners

        Talk Python To Me by Michael Kennedy

        Talk Python To Me

        579 Listeners

        Soft Skills Engineering by Jamison Dance and Dave Smith

        Soft Skills Engineering

        289 Listeners

        Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

        Super Data Science: ML & AI Podcast with Jon Krohn

        302 Listeners

        Python Bytes by Michael Kennedy and Brian Okken

        Python Bytes

        213 Listeners

        Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

        Syntax - Tasty Web Development Treats

        988 Listeners

        Darknet Diaries by Jack Rhysider

        Darknet Diaries

        8,088 Listeners

        Tech Brew Ride Home by Morning Brew

        Tech Brew Ride Home

        969 Listeners

        Practical AI by Practical AI LLC

        Practical AI

        200 Listeners

        AWS Podcast by Amazon Web Services

        AWS Podcast

        207 Listeners

        Django Chat by William Vincent and Carlton Gibson

        Django Chat

        75 Listeners

        Last Week in AI by Skynet Today

        Last Week in AI

        310 Listeners

        Machine Learning Street Talk (MLST) by Machine Learning Street Talk (MLST)

        Machine Learning Street Talk (MLST)

        100 Listeners

        The Pragmatic Engineer by Gergely Orosz

        The Pragmatic Engineer

        70 Listeners