Software Engineering Daily

Minecraft is a sandbox video game in which players build constructions out of 3-D cubes in a procedurally generated world. Minecraft is the best-selling PC game of all time. But Minecraft is not just a game. It is a platform for creativity, used by players within the game as well as programmers outside of it. Gabriel Simmer is a 16-year-old programmer who build NodeMC, a tool that wraps around theContinue reading...

Rust is a systems programming language being developed at Mozilla. Rust has features of a high-level functional language like Scala and a low-level, performance-driven language like C++. Steve Klabnik is a developer program member with Mozilla. In this episode, he discusses how Rust looks at memory management, type safety, mutability, and concurrency. We also dive into a discussion of the low level virtual machine, also known as the LLVM, whichContinue reading...

The process of building a software project requires us to make so many architectural decisions. Which programming languages should be used? Which cloud service provider? Which database? A newer type of building block is the analytics platform. Companies need to track events, aggregate metrics, and change the user’s experience based on aggregated data. Dan Kador is a co-founder of Keen IO, and he joins us to discuss the rise ofContinue reading...

At Netflix, 500 billion events and 1.3 petabytes of data are ingested by the system per day.  This includes video viewing activities, error logs, and performance events. On today’s episode, we dive deep into the data pipeline of Netflix, and how it evolved from their 1.0 version to the modern 2.0 version. Before listening to this episode, check out the blog post that inspired it. Sponsors IT Pro TV hasContinue reading...

Dropbox has been storing files on Amazon Web Services for 8 years, and Dropbox’s core business is storing files. For the past three years, Dropbox has been working on a project to migrate its file storage from Amazon Web Services to its own custom built infrastructure. Magic Pocket is the name of Dropbox’s new infrastructure layer, and it gives Dropbox more control and improved economics. James Cowling leads the storageContinue reading...

Distributed systems products are often marketed with terms like “real-time data” and “hassle-free scaling”, but what do those terms actually mean? Is data in a distributed system ever reliably “real time”? Do we ever have strong enough plans about our scalability strategy to say that scaling will be “hassle free”? Camille Fournier joins us today to discuss distributed systems in practice. Like everything in else in computer science, distributed systemsContinue reading...

Distributed databases are difficult to operate, and Crate.io wants to change that. Crate is a fast, scalable, easy-to-use SQL database that is built to run in containerized environments. An average software company runs several databases–MySQL for relational store, MongoDB for a document database, HDFS for blob storage and data warehouse, elastic search for search. On today’s show, Jodok Batlogg from Crate discuss the distributed architecture of Crate, and breaks downContinue reading...

Microsoft has built a suite of technologies on top of its Azure infrastructure as a service. Today, we discuss Azure Stream Analytics, a real-time event processing engine developed at Microsoft. Azure Streaming allows for constant querying of incoming data streams, and my guest Santosh Balasubramanian discusses Azure and the movement from batch processing to stream processing. Sponsors Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to getContinue reading...

Apache Spark is a framework for fast, distributed, in-memory analysis. Apache Cassandra is a distributed database management system that provides high availability and fast throughput. Today, we are collecting fast, big data streams from user behavior, smart phones and sensors, and the disk checkpointing of and query language of Hadoop MapReduce is no longer adequate.   Tim Berglund from Datastax came on Software Engineering Daily to explain how Apache CassandraContinue reading...

React started as just a view layer–it was the V in MVC. React has moved down the stack, with Flux, Redux, GraphQL, and Relay providing opinions for how React applications should structure their data flow. Jared Forsyth works at Khan Academy, which uses React on the front end. At Khan Academy, Jared has experimented with many different ways of handling data flow for a React application, and in today’s episodeContinue reading...