Machine learning is widely understood by the software community. But it is still hard to build a company around machine learning, because there is not easy access to large, unique data sets. Scale is a platform for training and validating data that is used for machine learning. Most machine learning models are built with supervised

Photo credit: Elisabeth Fall Ben Horowitz started Loudcloud with Marc Andreessen in 1999. He ran the company for eight years and chronicled his experience in his first book The Hard Thing About Hard Things. In his time running Loudcloud, the dotcom bubble burst, but Loudcloud needed cash so badly that he took the company public

Facebook leadership has a significant amount of engineers in its ranks, and engineers understand how to create an environment that appeals to other engineers.  Engineers do not like working on projects that they are not interested in, so Facebook optimizes for matching engineers to enjoyable work. Engineers do not like taking orders from managers, so

Chaos Monkeys is a book about Silicon Valley startups and Facebook. It is one of the most accurate books written about the modern technology industry, and captures both the negatives and the positives of software companies. Antonio Garcia Martinez is the author of Chaos Monkeys. He wrote the book after going through a gauntlet of

Facebook engineering is unique. Software is built at Facebook in a way that is distinctly different than any other company. In our series of shows about Facebook engineering, we have mostly covered the positive side of Facebook’s practices. In today’s show, we explore the downsides.  Facebook moves fast. Engineers within the company must move fast,

Peter Deng has worked on most of Facebook’s major products: Newsfeed, Instagram, Oculus, and Messenger. These different products have different requirements, but are all part of the same ethos of connecting people through social networks.  Facebook is a consumer product company that is powered by a strong engineering workforce. The relationship between product managers and

Jeff Rothschild was one of the earliest engineers to join Facebook. In the 1990s, Jeff had co-founded Veritas Software and helped it to its IPO in 2004. After Veritas, Jeff worked on several other of his own companies. He was working with Accel Partners on investments when he started to learn about Facebook. Accel was

In 2013, the Berkeley AMPLab was a center of innovation.  Three projects from AMPLab have turned into successful open source projects and companies: Spark, Mesos, and Alluxio. Haoyuan Li was the creator of Alluxio, and he returns to the show to discuss his journey taking Alluxio from a research project to a company that has

Redis is an in-memory database that persists to disk. Redis is commonly used as an object cache for web applications. Applications are composed of caches and databases. A cache typically stores the data in memory, and a database typically stores the data on disk. Memory has significantly faster access times, but is more expensive and

LinkedIn is a social network with petabytes of data.  In order to store that data, LinkedIn distributes and replicates that data across a large cluster of machines running the Hadoop Distributed File System. In order to run calculations across its large data set, LinkedIn needs to split the computation up using MapReduce-style jobs. LinkedIn has