The Real Python Podcast

Becoming More Effective at Manipulating Data With Pandas

Listen Later

Do you wonder if you’re taking the right approach when shaping data in pandas? Is your Jupyter workflow getting out of hand? This week on the show, Matt Harrison talks about his new book, “Effective Pandas: Patterns for Data Manipulation.”

Matt discusses working as a corporate consultant and migrating Excel users toward Python. We explore several “NumPy-isms” that beginners get stuck on. Matt shares advice about chaining operations in pandas, which some developers find controversial.

Course Spotlight: Sorting Data in Python With Pandas

In this video course, you’ll learn how to sort data in a pandas DataFrame using the pandas sort functions sort_values() and sort_index(). You’ll learn how to sort by one or more columns and by index in ascending or descending order.

Topics:

  • 00:00:00 – Introduction
  • 00:01:32 – Working as a consultant
  • 00:03:39 – Moving from Excel to Python
  • 00:06:50 – Who is the book for?
  • 00:10:15 – Using real data for examples
  • 00:16:16 – Sponsor: CData Software
  • 00:16:58 – What are patterns for data manipulation?
  • 00:18:38 – Cleaning and preparing data
  • 00:21:33 – What concepts were you most eager to share?
  • 00:26:57 – An example of chaining operations in pandas
  • 00:33:20 – NumPy-isms and other challenges in learning pandas
  • 00:40:20 – The use of exercises throughout the book
  • 00:43:50 – Video Course Spotlight
  • 00:45:01 – Challenges of using color throughout the book
  • 00:51:40 – Avoiding the slow path in pandas
  • 00:56:03 – What are you excited about in the world of Python?
  • 00:56:58 – What would you like to learn next?
  • 00:58:16 – Effective Pandas book
  • 00:58:38 – Social connections
  • 00:58:58 – Thanks and goodbye

    • Show Links:

    • Effective Pandas Digital Book Discount Link
    • Matt Harrison’s Site - MetaSnake
    • pandas - Python Data Analysis Library
    • Law of Demeter - Wikipedia
    • PyCon 2022 - Welcome to PyCon US
    • Production-ready Docker packaging for Python developers | Python=>Speed

      • Level up your Python skills with our expert-led courses:

      • Using pandas to Make a Gradebook in Python
      • The pandas DataFrame: Working With Data Efficiently
      • Sorting Data in Python With pandas

        • Support the podcast & join our community of Pythonistas

        ...more
        View all episodesView all episodes
        Download on the App Store
        The Real Python PodcastBy Real Python
        • 4.7
        • 4.7
        • 4.7
        • 4.7
        • 4.7

        4.7

        136 ratings

        More shows like The Real Python Podcast

        View all
        Software Engineering Radio - the podcast for professional software developers by se-radio@computer.org

        Software Engineering Radio - the podcast for professional software developers

        272 Listeners

        The Changelog: Software Development, Open Source by Changelog Media

        The Changelog: Software Development, Open Source

        283 Listeners

        Thoughtworks Technology Podcast by Thoughtworks

        Thoughtworks Technology Podcast

        41 Listeners

        Talk Python To Me by Michael Kennedy

        Talk Python To Me

        592 Listeners

        Software Engineering Daily by Software Engineering Daily

        Software Engineering Daily

        624 Listeners

        Soft Skills Engineering by Jamison Dance and Dave Smith

        Soft Skills Engineering

        269 Listeners

        Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

        Super Data Science: ML & AI Podcast with Jon Krohn

        298 Listeners

        Python Bytes by Michael Kennedy and Brian Okken

        Python Bytes

        213 Listeners

        Data Engineering Podcast by Tobias Macey

        Data Engineering Podcast

        142 Listeners

        Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

        Syntax - Tasty Web Development Treats

        982 Listeners

        DataFramed by DataCamp

        DataFramed

        266 Listeners

        Kubernetes Podcast from Google by Abdel Sghiouar, Kaslin Fields

        Kubernetes Podcast from Google

        181 Listeners

        Practical AI by Practical AI LLC

        Practical AI

        189 Listeners

        The Stack Overflow Podcast by The Stack Overflow Podcast

        The Stack Overflow Podcast

        64 Listeners

        The Pragmatic Engineer by Gergely Orosz

        The Pragmatic Engineer

        52 Listeners