Pandas is a swiss army knife for data processing in Python but it has long been difficult to customize. In the latest release there is now an extension interface for adding custom data types with namespaced APIs. This allows for building and combining domain specific use cases and alternative storage mechanisms. In this episode Tom Augspurger describes how the new ExtensionArray works, how it came to be, and how you can start building your own extensions today.

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Software development is a skill that can create value and reduce drudgery in a wide variety of contexts. Sometimes the causes that are most in need of software expertise are also the least able to pay for it. By volunteering our time and abilities to causes that we believe in, we can help make a tangible difference in the world. In this episode Eric Schles describes his experiences working on social justice initiatives and the types of work that proved to be the most helpful to the groups that he was working with.

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

One of the challenges of machine learning is obtaining large enough volumes of well labelled data. An approach to mitigate the effort required for labelling data sets is active learning, in which outliers are identified and labelled by domain experts. In this episode Tivadar Danka describes how he built modAL to bring active learning to bioinformatics. He is using it for doing human in the loop training of models to detect cell phenotypes with massive unlabelled datasets. He explains how the library works, how he designed it to be modular for a broad set of use cases, and how you can use it for training models of your own.

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Testing is a critical activity in all software projects, but one that is often neglected in data pipelines. The complexities introduced by the inherent statefulness of the problem domain and the interdependencies between systems contribute to make pipeline testing difficult to manage. To make this endeavor more manageable Abe Gong and James Campbell have created Great Expectations. In this episode they discuss how you can use the project to create tests in the exploratory phase of building a pipeline and leverage those to monitor your systems in production. They also discussed how Great Expectations works, the difficulties associated with pipeline testing and managing associated technical debt, and their future plans for the project.

The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA

We take it for granted every day, but creating and displaying vivid colors in our digital media is a complicated and often difficult process. There are different ways to represent color, the ways in which they are displayed can cause them to look different, and translating between systems can cause losses of information. To simplify the process of working with color information in code Thomas Mansencal wrote the Colour project. In this episode we discuss his motiviation for creating and sharing his library, how it works to translate and manage color representations, and how it can be used in your projects.

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA