Software Engineering Daily

When software is performing suboptimally, the programmer can use a variety of tools to diagnose problems and improve the quality of the code. A profiler is a tool for examining where a program is spending time.

Every program consists of a set of different functions. These functions call each other. The total amount of time that your program runs is the sum of the time your program spends in all of the different functions. When you run a program, you can execute a profiler on that program, and the profiler will give you a breakdown of which of the different functions time is being spent in.

If you have functions A, B, and C, your profiler might say that your program is spending 30% of its time in function A, 20% of its time in function B, and 50% of its time in function C.

Julia Evans is a software engineer at Stripe and the creator of a Ruby profiler called rbspy. rbspy can execute on a running Ruby program and report back with a profile. As Julia explains, a profiler turns out to be a non-trivial piece of software to build. To introspect a Ruby program, you need to understand how the Ruby interpreter is translating Ruby code into C structs for execution.

This episode is about profilers–but in order to talk about profilers, we also have to talk about Ruby, the Ruby interpreter, and the way that executing programs are laid out in memory.

The post Profilers with Julia Evans appeared first on Software Engineering Daily.

Message broker systems decouple the consumers and producers of a message channel. In previous shows, we have explored ZeroMQ, PubNub, Apache Kafka, and NATS. In this episode, we talk about another message broker: Apache Pulsar.

Pulsar is an open source distributed pub-sub message system originally created at Yahoo. It was used to scale products with high volumes of users–such as Yahoo Mail.

There are three components of a Pulsar deployment: the Pulsar broker (which handles the message brokering), Apache Bookkeeper (which handles the durable storage of the messages), and Apache Zookeeper, which manages the distributed coordination.

Lewis Kaneshiro joins the show to describe how Apache Pulsar works, and how it compares to other messaging systems like Apache Kafka. Lewis is the CEO of Streamlio, a company that builds messaging and stream processing systems for enterprises, and uses Pulsar in its core product.

The post Pulsar Messaging with Lewis Kaneshiro appeared first on Software Engineering Daily.

YouTube runs a large MySQL database to hold the metadata about its videos. As YouTube scaled, the database was sharded, and applications within YouTube had to write queries that were aware of the sharding layout of that database.

This is problematic because it pushes complexity to the application developer. An application developer shouldn’t have to be aware of how a database is laid out among different nodes. The developer should be able to issue a query and have the cluster simply return the data.

Vitess is an open source system for scaling large MySQL databases. Sugu Sougoumarane co-created Vitess at YouTube. Since YouTube is owned by Google, Vitess was able to leverage the Borg cluster manager developed at Google. Once Kubernetes came to market, it became more viable to make Vitess accessible to open source developers.

Sugu joins the show to talk about the scalability problems that YouTube’s database infrastructure encountered and the motivations for building Vitess.

The post Vitess: Scaling MySQL with Sugu Sougoumarane appeared first on Software Engineering Daily.

Social networks can make you feel connected to a global society. But those social networks are controlled by a corporate entity. The profit motivations of the corporation are not directly aligned with the experience of the users.

Mastodon is an open source, decentralized social network. Eugen Rochko started building Mastodon in response to his dissatisfaction with centralized social networks like Facebook and Twitter. In the Mastodon model, users can run their own nodes, and other users can connect to them. You can follow users whose accounts reside in other nodes.

Eugen joins the show to discuss how Mastodon works, and how its thousands of users interact on the platform. We explore the open source community that is building Mastodon, and speculate on the future of social networks.

The post Mastodon: Federated Social Network with Eugen Rochko appeared first on Software Engineering Daily.

Most technology companies rely on open source software projects. But open source software projects are often maintained by a group of people that is not affiliated with any particular company. When an open source project develops too much technical debt, it can become a tragedy of the commons. Who is responsible for maintaining these open source projects?

This is the motivation for open source bounties. Companies and individuals who rely on open source create bounties, which are financial incentives for developers to solve problems within the open source project.

Kevin Owocki is the creator of Gitcoin, a platform for open source bounties that is mediated by an Ethereum smart contract. Kevin joins the show to discuss his experience building Gitcoin–as well as some of the problems with the blockchain space, such as rampant ICOs. Gitcoin is NOT a cryptocurrency or token itself–it is a platform for open source software to be built more efficiently. Kevin was an awesome guest and you will enjoy the conversation.

Gitcoin is a nice example of a real-world Ethereum use case–it uses Ethereum for escrow: if I post a $25 bounty for someone to fix a bug in my open source project, I will lock up ether in a smart contract. When the bug is fixed, the programmer who fixed it will submit a pull request on Github, and I will release the ether from the smart contract to pay them.

We would love for you to fill out our listener survey at softwareengineeringdaily.com/survey. This will help us decide what other content to focus on.

Of course–you can also send me an email at any time, [email protected]. And in the meantime, if you are completely sick of cryptocurrencies, check out our back catalog of episodes at softwaredaily.com, or by downloading our Software Engineering Daily apps, which have all of our episodes including our Greatest Hits, which is a curated set of the most popular shows. The apps will soon have offline downloads and bookmarking.

The post Gitcoin: Open Source Bounties with Kevin Owocki appeared first on Software Engineering Daily.

You have probably missed some of the best episodes of Software Engineering Daily. If you listen to just a few episodes a week, it can be difficult to identify the high quality shows. And if you are new to the podcast, you have no idea how to find episodes that might appeal to you.

Software Engineering Daily has a discovery problem.

We have 600 episodes, and much of the content is evergreen. The shows we did a year ago on Apache Spark, or Ethereum, or ReactJS are still relevant today, and they get plenty of listens.

Keith and Craig Holliday built a recommendation system for Software Engineering Daily. Then they built a Software Engineering Daily iOS app to improve the experience of SE Daily listeners. You can use the SE Daily app to find the most popular episodes of this podcast, and to find episode recommendations based on what you have listened to.

In this episode, Keith and Craig join the show to explain why they built an app for Software Engineering Daily. You can find all the code for the SE Daily app at github.com/softwareengineeringdaily in case you want to fork it for your own podcast–or if you want to contribute to it.

Question of the Week: What is your favorite continuous delivery or continuous integration tool? Email [email protected] and a winner will be chosen at random to receive a Software Engineering Daily hoodie. 

The post Software Engineering Daily App with Keith and Craig Holliday appeared first on Software Engineering Daily.

Augmented reality is coming at us fast. Every large tech company is rumored to be building an AR product. Microsoft HoloLens is already available to developers. Pokemon Go, the most popular augmented reality product today, was made by a company that was spun out of Google. But Apple seems to be ahead of everyone.

Apple’s ARKit is a set of tools for developers to build augmented reality applications. The applications people are building with ARKit are remarkable, and two of those early adopters join the show today for an interview.

Jesse Bounds and Siyu Song work at Mapbox, a company that makes mapping, navigation, and location search SDKs. Location is natural companion to augmented reality. If I am walking down the street with a pair of augmented reality glasses on, those glasses can augment the world with information based on my location.

Because the fit between AR and mapping is so natural, Mapbox has been rapidly experimenting to build up an expertise in AR. As a result, Jesse and Siyu make for great guests to talk about what engineers can build with ARKit today and what might be possible in the future.

The post Augmented Reality with Jesse Bounds and Siyu Song appeared first on Software Engineering Daily.

Containers are widely used in projects that have adopted Docker, Kubernetes, or Mesos. Containers allow for better resource isolation and scalability. With all of the adoption of containers, companies like Red Hat, Google, and CoreOS are working on improved standards within the community.

Standards are important to this community because of its pace of growth and the number of concurrent projects. If you heard our recent episode about the Linux Kernel’s open source governance, you know that having some rules in place will help encourage the right kind of creativity to thrive.

In the world of containers, networking is not well addressed as it is highly environment specific. The Container Networking Interface is an effort to add specifications around how networks of containers can form.

Dan Williams is an engineer at Red Hat. In today’s episode, he explores the ideas behind the container networking interface, which gives insights into how the broader community of cloud native technologies is evolving as a whole.

The post Container Networking with Dan Williams appeared first on Software Engineering Daily.

The code in the Linux kernel changes all the time–11k lines are added, 5.8k lines are removed, and 2k lines are modified DAILY. Linux is an open source operating system that has been worked on for 25 years, and one reason the project is able to move so fast is its governance and release structure.

Greg Kroah-Hartman is a fellow at the Linux Foundation, where he takes part in many of the developments in the kernel. This episode was a dive into how open source software gets built at scale and what is in store for the future. The Kubernetes project has drawn comparable attention to the size of Linux, and the Kubernetes project is learning how to manage open source from the Linux community.

Check out our new topic feeds, in iTunes or wherever you find your podcasts. We’ve sorted all 500 of our old episodes into categories like business, blockchain, cloud engineering, JavaScript, machine learning, and greatest hits. Whatever specific area of software you are curious about, we have a feed for you. Check the show notes for more details.

Kernel Development

The post Linux Kernel Governance with Greg Kroah-Hartman appeared first on Software Engineering Daily.