Effective data science requires clean data. As data moves through the data pipeline, there may be errors introduced. Errors can also arise from code changes, database migrations, and other forms of data movement. How can you ensure data quality within a fast moving, dynamic data system? Datafold is a company built around data quality management.

Predicting the spread of COVID-19 is not easy. The best methods we have available require us to extrapolate trends from a large volume of data, and this requires the construction of large-scale models. Because of the expertise needed for developing these models, Silicon Valley engineers were brought in to help develop a maintainable model. Two

Databases are the source of truth for every company. Editing the data in the database normally requires writing a query in SQL or a domain specific querying language–languages that are only accessible to engineers and highly technical people.  BaseDash is a tool for interfacing with a database without requiring the usage of a query language.

Pachyderm is a system for data version control. Code has been version controlled for many years, but not data. In previous episodes with Joe Doliner, we explored the evolution of Pachyderm. In today’s show, we talk about the state of the company in 2020, as well as Pachyderm Hub, and end-to-end machine learning and data

Salesforce is a platform with a large number of developers, ISVs, and companies built on top of it. There is a thriving ecosystem of applications built and managed around Salesforce, leading to an important set of relationships and integration points between Salesforce and the other entities involved with the company. Kevin Poorman works at Salesforce

Twitter is a social media platform with billions of objects: people, tweets, words, events, and other entities. The high volume of information that gets created on Twitter everyday leads to a complex engineering problem for the developers building the Twitter search index. Nico Tonozzi is an engineer at Twitter. He joins the show to talk

Robinhood is a platform for buying and selling stocks and cryptocurrencies. Robinhood is complex, fast-moving, and financial, and together these things require high quality engineering in distributed systems, observability, and data infrastructure. Jaren Glover is an engineer at Robinhood, and he joins the show to talk about the problem space within Robinhood, as well as

The most popular email client is Gmail, the web-based email client from Google. Gmail is dominant, but that dominance has come at a price, namely speed. Gmail caters to the lowest common denominator, serving a large ecosystem of use cases and plugins. This makes for a slow overall performance. Superhuman is an email client built

The Internet Archive collects historical records of the Internet. The Wayback Machine is one tool from the Internet Archive which you may be familiar with. One project you may be unfamiliar with is book scanning. Internet Archive scans high volumes of books in order to digitize them. In today’s episode, Davide Semenzin joins the show

Biometric authentication uses signals from a human’s unique biology to verify identity. Forms of biometric authentication include fingerprints, eye patterns, and the way a person walks, otherwise known as gait. UnifyID is a company that builds systems for biometric authentication. John Whaley is the CEO of UnifyID, and he joins the show to talk through