Software Engineering Daily

Bot fraud, the New York tech scene, RethinkDB and open source; these topics and more are discussed in today’s episode. Two of the most popular guests return to the show to explore a variety of topics. Ben Halpern is the creator of The Practical Dev, a massively popular Twitter account and blog that you may recognize from its parody O’Reilly book covers. Haseeb Qureshi is an engineer at Airbnb, andContinue reading...

Google Compute Engine is the public cloud built by Google. It provides infrastructure- and platform-as-a-service capabilities that rival Amazon Web Services. Today’s guest Joe Beda was there from the beginning of GCE, and he was also one of the early engineers on the Kubernetes project. Google’s internal systems have made it easy for employees to spin up compute resources, but it was not a simple task to make this internalContinue reading...

Cloud computing was something much different in 2011, when Brian Gracely and Aaron Delp started The Cloudcast, a podcast I listen to on a regular basis. The Cloudcast features technical discussions about cloud infrastructure technology, and one of the most recent shows was a monologue by Brian Gracely where he explained his perspective on the industry rumblings about a Docker fork. The utility of a container for so many differentContinue reading...

The Simpsons is a classic, beloved television show. The scripts of The Simpsons have been made publicly available, and include dialogue, location, and character information. Todd Schneider used these scripts and other information sources as a corpus to analyze The Simpsons and find interesting statistics–such as who the most important supporting characters were, and how the ratings of the show have trended over time relative to other TV shows thatContinue reading...

Data engineering is the software engineering that enables data scientists to work effectively. In today’s episode, we explore the different sides of data engineering–the data science algorithms that need to be processed and the implementation of software architectures that enable those algorithms to run smoothly. The PANCAKE STACK is a 12-letter acronym that Chris Fregly gave to a collection of data engineering technologies including Presto, Cassandra, Kafka, Elastic Search, andContinue reading...

When a user of a social network updates her profile, that profile update needs to propagate to several databases that want to know about such an update–search indexes, user databases, caches, and other services. When Neha Narkhede was at LinkedIn, she helped develop Kafka, which was deployed at LinkedIn to help solve this very problem. Using Kafka as an event queue, LinkedIn adopted the CQRS architectural pattern together with eventContinue reading...

The intent of the DevOps movement is to get organizations moving faster and more effectively by breaking down siloes, and improving communication. Gene Kim’s book The Phoenix Project illustrated this by telling the fictional story of a company adopting a DevOps mentality. Although that book was fiction, Gene is an experienced engineer, having worked as founder and CTO of Tripwire, a software company that makes security and compliance automation software.Continue reading...

Building a monitoring system is a complex distributed systems problem. Events are produced from different points in an application and must be aggregated in order to form metrics. These events are often ingested by a time series database, which forms the backbone of our monitoring system. Theo Schlossnagle is the CEO of Circonus, where he has been working on architecting the company’s monitoring software for six years. In this episode,Continue reading...

In order to move software updates from the development team to production, companies do a variety of things. Some teams might email files to each other or use FTP or even floppy disks. Most companies today at least use version control systems like Git together with separate servers for development and production. When code is ready to move to production, a build that is on the development server gets copiedContinue reading...

Kafka Streams is a library for building streaming applications that transform input Kafka topics into output Kafka topics. In a time when there are numerous streaming frameworks already out there, why do we need yet another? To quote today’s guest Jay Kreps “the gap we see Kafka Streams filling is less the analytics-focused domain these frameworks focus on and more building core applications and microservices that process data streams.” JayContinue reading...