Software Engineering Daily

Git is a distributed file system for version control. Git is extremely reliable, fast, and secure, owing to the fact that it is one of the oldest pieces of open source software. But even battle-tested software can have vulnerabilities. In this episode, we explore a subtle git vulnerability that could have potentially led to git users executing malicious scripts when they intended to simply pull a repository.

Today’s guest Edward Thomson is a program manager at Microsoft, and a maintainer of libgit2, a C implementation of git. He also writes about git and hosts the podcast All Things Git. He is passionate about git development, which gave me a deeper perspective on something that I just consider a tool. But the only reason that tool is so good–the only reason it fades into the background–is because there are people that are passionate enough to work on it on a regular basis.

We also spent some time talking about the vulnerabilities that can spread through shared code environments–particularly in the realm of git, npm, and PHP. And we touched on how deployment workflows around git and Kubernetes are changing. Full disclosure: Microsoft, where Edward works, is a sponsor of Software Engineering Daily.

The post Git Vulnerability with Edward Thomson appeared first on Software Engineering Daily.

Shopify runs more than 600,000 small business websites. When Shopify was figuring out how to scale, the engineering teams did not have a standard workflow for how to deploy and manage services. Some teams used AWS, some teams used Heroku, some teams used other infrastructure providers.

To manage all those stores effectively, Shopify has built its own platform-as-a-service on top of Kubernetes called Cloudbuddies. Cloudbuddies was inspired by Heroku, and it allows engineers at Shopify to deploy services in an opinionated way that is perfect for Shopify.

Niko Kurtti is a production engineer at Shopify, and he joins the show to describe Shopify’s infrastructure–how they run so many stores, how they distribute those stores across their infrastructure, and the motivation for building their own internal platform on top of Kubernetes.

Shopify’s Journey to Kubernetes and PaaS: Niko Kurtti at QCon NY

The post Shopify Infrastructure with Niko Kurtti appeared first on Software Engineering Daily.

“Serverless” is a word used to describe functions that get deployed and run without the developer having to manage the infrastructure explicitly.

Instead of creating a server, installing the dependencies, and executing your code, the developer just provides the code to the serverless API, and the serverless system takes care of the server creation, the installation, and the execution. Serverless was first offered with the AWS Lambda service but has since been offered by other cloud providers.

There have also been numerous open source serverless systems. On SE Daily, we have done episodes about OpenWhisk, Fission, and Kubeless. All of these are built on the Kubernetes container management system. Kubernetes is an open-source tool used to build and manage infrastructure, so it is a useful building block for higher level systems.

Chad Arimura is the VP of serverless at Oracle, where he runs the Fn project, an open source serverless platform built on top of Kubernetes. In the past, he ran Iron.io, a message broker platform. Matt Stephenson also joins the show–he is a senior principal software engineer at Oracle and has experience from Amazon and Google, where he worked on Google App Engine (which was arguably one of the first “serverless” platforms).

We discussed why there are so many different serverless tools built on Kubernetes, and the tradeoffs that these serverless tools are exploring.

The post Function Platforms with Chad Arimura and Matt Stephenson appeared first on Software Engineering Daily.

When you interact with your bank, it probably feels different than when you interact with a software technology company. That’s because the biggest banks in the world were started before software became such a universally important tool. Their core competency is banking–not consumer software.

Today, most banks make consumer-facing software. But these banks were not founded by engineers. The software development process at a typical bank does not look like the software development process at a software company like Netflix.

Monzo is a digital bank that focuses on high quality engineering. Since it was started in 2015, Monzo has always thought of itself as a software company. This gives it certain advantages over older banks.

Today’s guest Richard Dingwall is an engineer at Monzo, and he joins the show to describe Monzo’s software architecture, the engineering strategy, and its migration to Kubernetes. Richard has prior experience at several different banks and financial institutions.

The post Build a Bank: Monzo with Richard Dingwall appeared first on Software Engineering Daily.

Rust is a systems programming language with a distinct set of features for safety and concurrency. In previous shows about Rust, we explored how Rust can prevent crashes and eliminate data races through its approach to type safety and memory management.

Rust’s focus on efficiency and safety makes it a promising language for networking code. Tokio is a set of networking libraries built on Rust. Tokio enables developers to write asynchronous IO operations by way of its multithreaded scheduler. Tokio’s goal is to make production-ready clients and servers easy to create by focusing on small, reusable components.

Carl Lerche is an engineer at Buoyant, a company that makes the popular Linkerd and Conduit service mesh systems. Kubernetes developers deploy service mesh to their distributed applications as sidecar proxies. These proxies need to be low-latency and highly reliable. In that light, it makes sense that Conduit (the more recent service mesh from Buoyant) is built using Rust. Carl joins the show to describe why Rust is useful for building networked services.

The post Rust Networking with Carl Lerche appeared first on Software Engineering Daily.

A database stores data to an underlying section of storage. If you are an application developer, you might think of your persistent storage system as being the database itself–but at a lower level, that database is writing to block storage, file storage, or object storage.

A container orchestration system manages application containers. If you want to run WordPress (a blogging platform) on Kubernetes, that means you also need to run a database to store your blog posts in a persistent way. To run a database, you need to have an underlying storage medium–which could be a disk that is at your on-prem data center, or block storage on a disk at a cloud provider.

Kubernetes is not the only container orchestrator. There’s also Cloud Foundry, Mesos, Docker Swarm, and several others. Each of these container orchestrators needs to be able to run a variety of persistent workloads (such as a MySQL database or a Kafka cluster). Each of these persistent workloads needs to be able to use different types of backing storage.

With the range of container orchestrators and the range of backing storage types, a problem arises. Every storage type would have to write custom code to connect to each container orchestrator.

The solution to this is the CSI: the container storage interface. The CSI is an interface layer between the container orchestrator and the backing storage system. In today’s episode, Jie Yu from Mesosphere describes the motivation for the CSI, and gives an overview for its design principles. There are great lessons here for anyone working with containers or distributed systems in general.

Stripe: Designing Robust and Predictable APIs with Idempotency 

The post Container Storage with Jie Yu appeared first on Software Engineering Daily.