Software Engineering Daily

Upcoming events:

A Conversation with Haseeb Qureshi at Cloudflare on April 3, 2019

FindCollabs Hackathon at App Academy on April 6, 2019

Google’s Borg system is a cluster manager that powers the applications running across Google’s massive infrastructure. Borg provided inspiration for open source tools like Apache Mesos and Kubernetes.

Over the last decade, some of the largest new technology companies have built their own systems that fulfill the roles of cluster management and resource scheduling. Netflix, Twitter, and Facebook have all spoken about their internal projects to make distributed systems resource allocation more economical. These companies find themselves continually reinventing scheduling and orchestration, with inspiration from Google Borg and their own internal experiences running large numbers of containers and virtual machines.

Uber’s engineering team has built a cluster scheduler called Peloton. Peloton is based on Apache Mesos, and is architected to handle a wide range of workloads: data science jobs like Hadoop MapReduce; long running services such as a ridesharing marketplace service; monitoring daemons such as Uber’s M3 collector; and database services such as MySQL.

Min Cai and Mayank Bansal are engineers at Uber who work on Peloton. When they set out to create Peloton, they looked at the existing schedulers in the ecosystem, including Kubernetes, Mesos, Hadoop’s YARN system, and Borg itself.

Both Min and Mayank join the show today to give a brief history of distributed systems schedulers and discuss their work on Peloton. They have been working in the world of distributed systems schedulers for many years–including experiences building core Hadoop infrastructure and virtual machine schedulers at VMware.

The post Peloton: Uber’s Cluster Scheduler with Min Cai and Mayank Bansal appeared first on Software Engineering Daily.

Cloud computing started to become popular in 2006 with the release of Amazon EC2, a system for deploying applications to virtual machines sitting on remote data center infrastructure . With cloud computing, application developers no longer needed to purchase expensive server hardware. Creating an application for the Internet became easier, cheaper, and simpler.

As the cloud has become popular, new ways of deploying applications have emerged. A developer with a web app has so many different options.

You can host your app on an Amazon EC2 server, which will require you to manage cloud infrastructure in case your server crashes. You can deploy your app to Heroku, which gives your cloud deployment better uptime guarantees for a higher price than Amazon EC2. Or you can use Linode, or Microsoft Azure, or Google Cloud.

There is such a large market for cloud computing that the world of cloud providers serves more niches every year. In past episodes we have explored a variety of different cloud providers, and the markets they target.

Pivotal Cloud Foundry is for managing complex distributed systems applications, typically with large teams. Firebase is a cloud provider that simplifies the developer experience for applications with small teams. Spotinst is a cloud provider that emphasizes low cost. Zeit is a cloud provider that is built to manage applications through serverless “functions-as-a-service” like AWS Lambda.

In today’s episode, Mathias Biilman Christensen, CEO of Netlify, joins the show. Netlify is a cloud provider that was built for modern web projects. Netlify represents the convergence of several trends in software development converging: static site deployment, serverless functions, a desire to have a “no-ops” deployment with minimal management, and the rise of newer tools like GraphQL and Gatsby.

Mathias explores these trends in detail, and explores the technical challenges of building Netlify. He was a great guest, capable of talking about difficult backend problems that require writing C++, as well as the frontend world of JavaScript frameworks.

One announcement before we begin: we are having a $5000 hackathon. The $5000 hackathon is for a new product we’ve been working on: FindCollabs. FindCollabs is a platform for finding collaborators and building projects. Whether you are an engineer, a musician, a designer, a videographer, or an artist, FindCollabs lets you find people and collaborate. To try out FindCollabs, just go to FindCollabs.com, you can make a project or you can join someone else’s project. And it’s very easy to make these projects–you don’t need to have anything built yet–you need to have a vision for what you want to build. And to find out about the hackathon, go to findcollabs.com/hackathon. We are giving away $5000 in cash to the coolest projects that get built before Sunday April 14th. So I recommend getting started early, finding some people to collaborate with, and building some cool stuff!

The post Netlify with Mathias Biilmann Christensen appeared first on Software Engineering Daily.

A Kubernetes cluster presents multiple potential attack surfaces: the cluster itself, a node running on the cluster, a pod running in the node, a container running in a pod. If you are managing your own Kubernetes cluster, you need to be aware of the security settings on your etcd, your API server, and your container build pipeline.

Many of the security risks of a Kubernetes cluster can be avoided by using the default settings of Kubernetes, or by using a managed Kubernetes service from a cloud provider or an infrastructure company. But it is useful to know about the fundamentals of operating a secure cluster, so that you can hopefully avoid falling victim to the most common vulnerabilities.

Liz Rice wrote the book Kubernetes Security with co-author Michael Hausenblas. Liz works at Aqua Security, a company that develops security tools for containerized applications. In today’s show, Liz gives an overview of the security risks of a Kubernetes cluster, and provides some best practices including secret management, penetration testing, and container lifecycle management.

The post Kubernetes Security with Liz Rice appeared first on Software Engineering Daily.

Cloud computing has been popular for less than twenty years. Large software companies have existed for much longer. If your company was started before the cloud became popular, you probably have a large, data center on your companies premises. The shorthand term for this software environment is “on-prem”.

Deploying software to your own on-prem servers can be significantly different than deploying to remote servers in the cloud. In the cloud, servers and resources are more standardized. It is often easier to find documentation and best practices for how to use cloud services.

Many of the software vendors who got started in the last decade created their software in the cloud. For example, Readme.io makes it easy for companies to create hosted documentation. Their early customers were startups and other cloud-native companies. All of those companies were happy to consume the software in the cloud. As time went on, Readme found that other customers wanted to use the Readme product as a self-hosted, on-prem service. Readme needed to figure out how to deploy their software easily to the “on-prem” environment.

It turns out that this is a common problem. Software vendors who want to sell to on-prem enterprises must have a defined strategy for making those deployments to on-prem infrastructure–and those deployments are not always easy to configure.

Replicated is a company that allows cloud-based software to easily deploy to on-prem infrastructure. Grant Miller is the founder of Replicated and he joins the show to discuss on-prem, cloud, and the changing adoption patterns of enterprise software companies.

The post Replicated: On-Prem Deployments with Grant Miller appeared first on Software Engineering Daily.

Infrastructure software is having a renaissance.

Cloud providers offer a wide range of deployment tools, including virtual machines, managed Kubernetes clusters, standalone container instances, and serverless functions. Kubernetes has standardized the container orchestration layer and created a thriving community. The Kubernetes community gives the cloud providers a neutral ground to collaborate on projects that benefit everyone.

The two forces of cloud providers and Kubernetes have led to massive improvements in software quality and development practices over the last few years. But one downside of the current ecosystem is that many more developers learn how to operate a Kubernetes cluster than perhaps is necessary. “Serverless” tools are at a higher level than Kubernetes, and can improve developer productivity–but a risk of using a serverless tool is the potential for lock-in, and a lack of portability.

Knative is an open-source serverless platform from Google built on top of Kubernetes. Ville Aikas is a senior staff engineer at Google who has worked at the company for eleven years. With his experience, Ville brings a rare perspective to the subjects of Kubernetes, serverless, and the infrastructure lessons of Google. Ville joins the show to discuss Knative, the motivation for building it, and the future of “serverless” infrastructure.

The post Knative: Serverless Workloads with Ville Aikas appeared first on Software Engineering Daily.

Virtualization software allows companies to get better utilization from their physical servers. A single physical host can manage multiple virtual machines using a hypervisor. VMware brought virtualization software to market, creating popular tools for allowing enterprises to deploy virtual machines throughout their organization.

Containers provide another improvement to server utilization. A virtual machine can be broken up into containers, allowing multiple services to run within a single VM. Containers proliferated after the popularization of Docker, and the Kubernetes open source container orchestration system grew to be the most common way of managing the large numbers of containers that were running throughout an organization.

As Kubernetes has risen to prominence, software infrastructure companies have developed Kubernetes services to allow enterprises to use Kubernetes more easily. VMware’s PKS is one example of a managed Kubernetes service.

Brad Meiseles is a senior director of engineering at VMware with more than nine years of experience with the company. He joins the show to discuss virtualization, Kubernetes, containers, and the strategy of a large infrastructure provider like VMware.

The post VMware Kubernetes Strategy with Brad Meiseles appeared first on Software Engineering Daily.

HashiCorp was founded seven years ago with the goal of building infrastructure tools for automating cloud workflows such as provisioning, secret management, and service discovery. Hashicorp’s thesis was that operating cloud infrastructure was too hard: there was a need for new tools to serve application developers.

Hashicorp founders Mitchell Hashimoto and Armon Dadgar began releasing open source tools to fulfill their vision of better automation. Terraform, Vagrant, Consul, and other tools created by Hashicorp gained popularity, and Hashicorp began iterating on their business model. Today, Hashicorp makes money by offering enterprise features and support to enterprises such as Pinterest, Adobe, and Cruise Automation.

Over the last seven years, enterprise software infrastructure has changed rapidly. First, enterprises moved from script-based infrastructure automation to container orchestration frameworks. Then, the container orchestration world consolidated around Kubernetes.

Today, large enterprises are rapidly adopting Kubernetes with a mix of public cloud and on-prem vendors. At the same time, these enterprises are also becoming more willing to consume proprietary tools from the public cloud providers.

Hashicorp has benefitted from all of this change. Their different tools fit into a variety of workflows, and are not closely coupled with any particular cloud provider or platform solution.

Armon and Mitchell join today’s show to discuss the business model and the product philosophy of HashiCorp. We also touch on service mesh, zero trust networking, and their lessons from the container orchestration wars.

The post Scaling HashiCorp with Armon Dadgar and Mitchell Hashimoto appeared first on Software Engineering Daily.

Prometheus is an open source monitoring system and time series database. Prometheus includes a multi-dimensional data model, a query language called PromQL, and a pull model for gathering metrics from your different services. Prometheus is widely used by large distributed systems deployments such as Kubernetes and Cloud Foundry.

Prometheus gathers metrics from your services by periodically scraping those services. Those metrics get gathered, compressed, and stored onto disk for querying. But Prometheus is designed to store all of its records on one host in one set of files–which limits the scalability and availability of those metrics.

Cortex is an open source project built to scale Prometheus. Cortex effectively shards Prometheus by parallelizing the “ingestion” and storage of Prometheus metrics. Cortex can take metrics from multiple Prometheus instances and store them across a distributed NoSQL database like DynamoDB, BigTable, or Cassandra.

Bryan Boreham is an engineer at Weaveworks, where he works on deployment, observability, and monitoring tools for containers and microservices. He wrote much of the code for Cortex, and we met up at KubeCon North America to talk about the motivation for creating Cortex, the broader landscape of Kubernetes monitoring, and other approaches to scaling Prometheus.

The post Prometheus Scalability with Bryan Boreham appeared first on Software Engineering Daily.

When a developer provisions a cloud server, that server is called an “instance”. These instances can be used for running whatever workload a developer has, whether it is a web application, a database, or a set of containers.

The cloud is cheap to get started on. New applications with few users can often be hosted on infrastructure that is less than $10 per month. But as an application grows in popularity, there is more demand for CPUs and storage. A company will start to buy more and more servers to scale up to the requirements of their growing user base. The costs of running infrastructure in the cloud will increase, and the company will start to look for ways to save money.

One common method of saving money is to buy “spot instances”. A spot instance is an instance that is cheaper than “reserved instances” or “on-demand” instances. The reason that there are different instance types is because a giant cloud provider has a highly variable amount of work that is being demanded from that cloud provider.

If you are in charge of AWS, you have to make sure that at any given time, you can give server resources to anyone that asks for it. Your data centers need to have physical machines that are ready to go at any time. This means that much of the time, you have server resources that are going unused.

If you are a cloud provider, how can you get people to use your compute resources? You can make them cheaper. So a user can come along and buy your compute at the discounted “spot” price.

But this presents a problem for the cloud provider. If you start to give away your compute at cheaper prices, and then the overall demand for your cloud resources go up once again, you are going to miss out on profits. As the cloud provider, you need to kick people off of your spot instances, so that you can take those same instances and sell them to people at the higher market prices.

And this presents a problem for the user. If you buy a cheap spot instance, that instance is only available until the cloud provider decides to kick you off. You have a tradeoff between cost and availability of your instances. Because of this, spot instances are typically used only for workloads that are not mission critical–workloads that can afford to fail.

Spotinst is a company that allows developers to deploy their workloads reliably onto spot instances. Spotinst works by detecting when a spot instance is going to be reclaimed by a cloud provider and re-scheduling the workload from that cloud provider onto a new spot instance.

The post Spot Instances with Amiram Shachar appeared first on Software Engineering Daily.

Chinese Internet companies operate at a massive scale.

WeChat has over a billion users and is widely used as the primary means of payment by urban Chinese consumers. Alibaba ships 12 million packages per day, which is four times the amount of Amazon. JD.com, a Chinese ecommerce company, has perhaps the largest production Kubernetes installation in the world.

China’s rapid adoption of Internet services, combined with a large population and a growing middle class has led to the creation of Internet giants on par with the social networks, ecommerce sites, and ridesharing startups of the United States.

Last November, I attended the first KubeCon China and saw firsthand how the Chinese Internet companies are using open source software to scale their infrastructure.

Despite the differences between the US and China, the culture of technologists at KubeCon felt familiar. In some ways, it was just like any other Kubernetes conference that I have attended: large numbers of engineers trying to find the cutting edge of technology, and learning how to solve the problems they are facing back at the office.

There were presentations on scaling databases and service meshes and machine learning on Kubernetes. Outside of these presentation halls, there were tables where you could pick up a translation device so that Chinese-only and English-only presentations could be understood by the other nationality.

Dan Kohn joins the show to talk about Chinese Internet companies and how they are adopting Kubernetes. Dan is the executive director of the Cloud Native Computing Foundation, an organization within the Linux Foundation that organizes KubeCon. Before joining the CNCF, Dan worked as an entrepreneur, engineer, and executive at several technology companies.

Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

The post Kubernetes in China with Dan Kohn appeared first on Software Engineering Daily.