The data lake architecture has become broadly adopted in a relatively short period of time.  In a nutshell, that means data in it’s raw format stored in cloud object storage.  Modern software and data engineers have no shortage of options for accessing their data lake, but that list shrinks quickly if you care about features like transactions.  Apache Hudi is a platform for building streaming data lakes that is optimized for lake engines and batch processing.  In this episode, I interview Vinoth Chandar, creator of the Hudi Project and Founder and CEO at Onehouse.

Sponsorship inquiries: [email protected]

The post Apache Hudi with Vinoth Chandar appeared first on Software Engineering Daily.

A data catalog provides an index into the data sets and schemas of a company. Data teams are growing in size, and more companies than ever have a data team, so the market for data catalog is larger than ever.

Mark is the CEO of Stemma and the co-creator of Amundsen, a data catalog that came out of Lyft. We have previously explored the basics of Amundsen. In today’s episode, Mark Grover returns to the show to talk about the art and science of data catalogs.

Sponsorship inquiries: [email protected]

The post Data Catalog in Practice with Mark Grover appeared first on Software Engineering Daily.

Splunk is a monitoring and logging platform that has evolved over its 18 years of existence. In its modern focus on observability it is focused on open source and AIOps. Observability has evolved with the growth of Kubernetes, and Splunk’s work around OpenTelemetry has kept parity with the open source community of Kubernetes.

Spiros Xanthos is the general manager of observability at Splunk. He joins the show to talk about Splunk’s modern product portfolio and his work on his own company prior to being acquired by Splunk.

Sponsorship inquiries: [email protected]

The post Splunk Platform with Spiros Xanthos appeared first on Software Engineering Daily.

Barry McCardel

Co-Founder and CEO at Hex

Caitlin Colgrove

Co-Founder and CTO at Hex

In contrast to other IDEs, the notebook interface offers software developers a unique environment idealized for data professionals.  Despite the growth in popularity, a surprising learning curve still exists for setup and configuration.  A siloed notebook offers no native collaboration tools.  While one can connect to a SQL database programmatically, if you’re looking for an ideal ergonomic environment for some heavy duty SQL queries, many developers seek an external tool for that job.

In this episode, Kyle interviews Barry McCardel and Caitlin Colgrove from Hex.  Hex is a collaborative data workspace that makes it easy to go from idea to analysis to sharing.  We talk about Hex’s offering and the evolving space of notebook solutions for going beyond some of the issues noted above.

The post Hex Collaborative Data Workspace with Barry McCardel and Caitlin Colgrove appeared first on Software Engineering Daily.

When writing code, test driven development is a common accepted methodology to ensure the development of high quality software.  Your organization’s data, on the other hand, is an entirely different challenge.  Data can be missing due to human error, a failure with a 3rd party provider, a botched release, or dozens of other issues.  When not missing, data can still become corrupted or start exhibiting a trend in the wrong direction that isn’t obvious to notice.

Anomalo is a complete data quality platform.  It can monitor your enterprise data and alert you to problems that are automatically detected.  In this episode, I interview Jeremy Stanley about the ways in which teams are using the platform to monitor and improve their data quality.

Sponsorship inquiries: [email protected]

The post Data Quality Using Anomalo with Jeremy Stanley appeared first on Software Engineering Daily.

Database product companies typically have a few phases. First, the company will develop a technology with some kind of innovation such as speed, scalability, or durability. The company will offer support contracts around that technology for a period of time, before eventually building a managed, hosted offering.

PlanetScale is a database company built around the Kubernetes-based Vitess technology. Sugu Sougoumarane is the CTO of PlanetScale, and formerly worked at YouTube, where he built large-scale SQL databases. In today’s show, he talks about building out the core PlanetScale technology, tuning PlanetScale’s consensus model, and the development of the hosted cloud service offering of the company.

Sponsorship inquiries: [email protected]

The post Scaling PlanetScale with Sugu Sougoumarane appeared first on Software Engineering Daily.

Couchbase is a distributed NoSQL cloud database. Since its creation, Couchbase has expanded into edge computing, application services, and most recently a database-as-a-service called Capella. 

Couchbase started as an in-memory cache and needed to be rearchitected to be a persistent storage system. In this episode, I interview Ravi Mayuram, SVP Products and Engineering at Couchbase about the architecture and the history of Couchbase. To learn more about Couchbase, check out Couchbase.com/SEDaily.

Sponsorship inquiries: [email protected]

The post Couchbase Architecture with Ravi Mayuram appeared first on Software Engineering Daily.

If you haven’t encountered a data quality problem, then you haven’t yet worked on a large enough project.  Invariably, a gap exists between the state of raw data and what an analyst or machine learning engineer needs to solve their problem.  Many organizations needing to automate data preparation workflows look to Trifacta as a solution.  In this episode, I interview Joseph Hellerstein, professor of Computer Science at UC Berkeley and co-founder of Trifacta.

Sponsorship inquiries: [email protected]

The post Trifacta with Joe Hellerstein appeared first on Software Engineering Daily.

Relational databases have been a fixture of software applications for decades.  They are highly tuned for performance and typically offer explicit guarantees like transactional consistency.  More recently, there’s been a figurative cambrian explosion of other-than-relational databases.  Simple key value stores or counters were an early win in this space. Managing a graph data structure is a more challenging task than key value stores.  Asking questions of graphs demands a robust query language.  Ideally, you’d also want to provide common graph algorithms to users out of box.

MemGraph is an in-memory graph database that seeks to fill this need in a real-time fashion.  It’s a performance driven solution that offers developers a mature option for managing and utilizing graph data.  In this interview, I speak with Dominik Tomicevic, Founder and CEO of MemGraph.  We discuss implementation and adoption details of MemGraph as well as the ways machine learning engineers use it for feature engineering in real time.

Sponsorship inquiries: [email protected]

The post MemGraph with Dominik Tomicevic appeared first on Software Engineering Daily.

The lifeblood of most companies is their sales departments.  When you’re selling something other than a commodity, it’s typically necessary to carefully groom the onboarding experience for inbound future customers.  Historically, companies approached this in a one-size-fits-all manner, giving all customers a common experience.

In today’s data-driven age, a better experience can be provided that is data-driven, personalized, and automated.  Amplemarket leverages machine learning in the development of next-generation sales tools.  In this episode, I interview João Batalha, Co-Founder, and CEO of Amplemarket.

Sponsorship inquiries: [email protected]

The post Amplemarket with João Batalha appeared first on Software Engineering Daily.