Software Engineering Daily

Customer data pipelines power the backend of many successful web platforms. In a customer data pipeline, data is collected from sources such as mobile apps and cloud SaaS tools, transformed and munged using data engineering, stored in data warehouses, and piped to analytics, advertising platforms, and data infrastructure. RudderStack is an open source customer data

Managing Kubernetes nodes leads to operational complexity, security issues, and nodes that are perhaps more expensive to run than necessary. Deferring the node management to an underlying platform abstracts away these problems and can improve operations. Madhuri Yechuri runs Elotl, a nodeless Kubernetes platform. She joins the show to talk about the architecture and purpose

Retool is a company that allows customers to build complex internal tools using a high level GUI. Users configure the relationships between these different tools, giving them the ability to build applications even without much background in engineering. Of course, having some engineering expertise helps, as users can write JavaScript to interact with the higher

The software supply chain consists of packages, imports, dependencies, containers, and APIs. These different components each have unique security risks. To ensure the security of their software supply chain, many developers use tools to analyze and scan their infrastructure for vulnerabilities. Barak Schoster works at Bridgecrew, a DevSecOps cloud security platform. He joins the show

The data lake architecture has become broadly adopted in a relatively short period of time.  In a nutshell, that means data in it’s raw format stored in cloud object storage.  Modern software and data engineers have no shortage of options for accessing their data lake, but that list shrinks quickly if you care about features

National Instruments develops software and hardware for engineering in a wide variety of domains, from aerospace to government technology to application testing. The interface between hardware and software presents a variety of difficult engineering challenges. Luke Schreier is a Senior Vice President at National Instruments and joins the show to discuss the engineering and management

Automating video search requires a data pipeline that extracts metadata from videos and allows users to annotate the video with information that correlates with that metadata. The video needs to be segmented into intervals that define the search space, and the search space needs to be queryable by a user. Sieve Data is an automated

Increasingly, technology groups need to be strategic about the cloud services they adopt to ensure their vendor’s pricing is both fair and measured exactly right for the application’s unique access pattern.  Definitely answering questions like these can take a significant amount of time and energy from your most valuable engineers.  Taloflow is a company trying

Developers struggle with mental health, and this struggle has only gotten more acute during the pandemic. Sitting in front of a computer all day, engulfed by social media and code can cause us to lose our sanity. I personally had some issues that were hard to grapple with during the COVID lockdowns. In today’s show,

A data catalog provides an index into the data sets and schemas of a company. Data teams are growing in size, and more companies than ever have a data team, so the market for data catalog is larger than ever. Mark is the CEO of Stemma and the co-creator of Amundsen, a data catalog that came out of