Software Engineering Daily

Servers in a data center fail. Sometimes entire data centers have a power outage. Bugs in an application make it into production. Human operators make mistakes and cause data to be deleted.

Failure is unavoidable. We make backups and replicate our servers so that when a failure occurs, we can quickly respond to it without making the user feel much pain. But how can we test that our response will work before an actual catastrophe occurs?

Kolton Andrus is CEO of Gremlin, a company that works on failure injection as a service. Gremlin is based on ideas around planned failure that Kolton learned from his years at Amazon and Netflix.

We ended up talking as much about the culture of Netflix and Amazon as we did about how and why to build failure injection. It’s always nice to share war stories with other people who have worked at Amazon because the culture is so distinct. If you want to know more about Amazon’s culture, check out the episode tomorrow with Brad Stone, author of The Everything Store.

The post Failure Injection with Kolton Andrus appeared first on Software Engineering Daily.

If you wanted to build a machine learning model to understand human health, where would you get the data? A hospital database would be useful, but privacy laws make it difficult to disclose that patient data to the public. In order to publicize the data safely, you would have to anonymize it, so that a patient’s identity could not be derived from data about that patient–and true anonymization is notoriously difficult.

In every industry where privacy is a concern there is a similar challenge. If there is no place with public data sets, there is no place where the machines can go to learn. The possible machine learning applications that we can build are limited by the data sets that are available.

Auren Hoffman started his company SafeGraph to unlock data sets so that machine learning algorithms can learn from that data. In this episode, we talk about the machine learning landscape in both the short- and long-term time horizons. We also discussed some of Auren’s strategies for building companies, which have been crucial for me in thinking about how to build Software Engineering Daily.

Where Should Machines Go to Learn?

Quoracast Episode

The post Where Machines Go to Learn with Auren Hoffman appeared first on Software Engineering Daily.

Most tech companies are moving toward a highly distributed microservices architecture. In this architecture, services are decoupled from each other and communicate with a common service language, often JSON over HTTP. This provides some standardization, but these companies are finding that more standardization would come in handy.

At the ridesharing company Lyft, every internal service runs a tool called Envoy. Envoy is a service proxy. Whenever a service sends or receives a request, that request goes through Envoy before meeting its destination.

Matt Klein started Envoy, and he joins the show to explain why it is useful to have this layer of standardization between services. He also gives some historical context for why Envoy was so helpful to Lyft.

The post Service Proxying with Matt Klein appeared first on Software Engineering Daily.

Cloud computing has pushed computation away from our own private servers and into virtual machines running on a data center. In the world of cloud computing, processing is centralized in these data centers, and our smartphone and laptop application performance suffers from having high latency between the client and the cloud server.

As machine learning proliferates, the current model of cloud computing will become too slow. A small difference in the time it takes to refresh a machine learning model for a drone or car could be the difference between life and death.

Computation will move to the edge. The same drones, cars, and IoT devices that need their models updated quickly will form a peer-to-peer network with which to distribute time-sensitive tasks. One bellwether for this real time peer-to-peer network might be Uber’s Ringpop, a fault-tolerant application layer sharding system.

In such an edge computing model, your device could federate a complex request out to other nearby devices that have spare processing capacity, pay for those compute cycles using Bitcoin, and receive a response without any request to a centralized cloud server. The cloud servers would still be around, but they would be responsible for doing offline computation across large data sets.

This prediction was described by Peter Levine, a partner at Andreessen Horowitz in his talk “The End of Cloud Computing”. In this episode, Peter discusses the pressures that our pushing toward edge computing and away from the cloud.

The post The End of Cloud Computing with Peter Levine appeared first on Software Engineering Daily.

What is the relationship between your brain and your conscious experiences? This is is the fundamental question of the work of Donald Hoffman, a professor of computer science and cognitive science at UC Irvine.

When Hoffman was a child, he wondered whether there was a cognitive dividing line between humans and machines, and that curiosity has driven him to his current work–building a mathematical framework which we can use to model consciousness.

Humans would like to believe that evolution selects for traits that allow us to understand the world as it is. From his experimental simulations, Hoffman has shown the opposite–that evolution pushes us towards a mistaken version of reality.

For listeners who are interested in theories about whether we live in a simulation, this episode is for you. If you are skeptical of the simulation theory then this episode will also be useful for you, as Hoffman gives one of the most nuanced, comprehensive explanations of reality that I have heard.

TED Talk

The Case Against Reality

The post Reality with Donald Hoffman appeared first on Software Engineering Daily.

Engineers today have a variety of career options. You could go work for a large corporation, you could raise money and start a startup, you could freelance and move from job to job with freedom–or you could start a business with the goal of quickly becoming profitable.

Courtland Allen was a guest on Software Engineering Daily a few months ago, when he discussed Indie Hackers, a platform he built to share the stories of engineers building business on their own and making money.

We only touched the tip of the iceberg in our conversation, so I was excited to invite him to the first Software Engineering Daily Meetup, which occurred earlier this month. Today we are republishing his talk, and I would love to hear your feedback on this format. We will be experimenting more with new hosts and formats throughout 2017, and if you have ideas for the show or you are interested in hosting a show, please send me an email.

Also–the first Software Engineering Daily Meetup was fantastic–there were ~200 people showing up so we may have to cap attendance on the next one. Please join the Meetup group if you are interested, and we will let you know when we schedule our next event.

Courtland Allen slides in PDF.

The post Making Money Online for Software Engineers with Courtland Allen appeared first on Software Engineering Daily.

In the 1990s, the barriers to starting a company were significant. Not only did you need an idea, you needed $200,000 for servers and Oracle licenses. With cloud computing, the up-front financial costs of getting a company off the ground have been mostly eliminated–but the idea of starting a company is still perceived as risky.

The process of building software has changed dramatically in the last twenty years, but many of the challenges of managing a software company remain timeless. How do you hire properly? How do you keep track of a software stack that is growing in complexity? How do you handle dissenting opinions from employees?

Mike Wolfe has been building software companies since the 1990s and joins the show to discuss how to build products and manage engineering teams. It’s a wide-ranging discussion including technological trends, interpersonal skills, and startup financing.

The post Startup Engineering with Mike Wolfe appeared first on Software Engineering Daily.

When a human passes away, we create a tombstone as a memorial. Friends and family visit a grave to remember the times they had with that person while they were still alive. Memorial bots are another way to celebrate the life of someone who has passed away. A memorial bot is created by taking the messages sent by a deceased person and passing it through a machine learning model in order to make a bot that replicates the deceased person.

Eugenia Kuyda is the CEO of Luka, a company that builds AI products. When her friend Roman Mazurenko suddenly died, she worked with her team to make a bot that replicates his speech patterns. In our interview, we discussed memorial bots, deep learning, and the product Luka is working on–Replika, a personal AI friend for anyone.

The post Bot Memorial with Eugenia Kuyda appeared first on Software Engineering Daily.

When you are deciding who to marry, you are using an algorithm. The same is true when you are looking for a parking space, playing a game of poker, or deciding whether or not to organize your closet. Algorithms To Live By is a book about the computer science of human decisions. It offers strategies for how to think through everyday life like a computer scientist.

Brian Christian has a background in computer science and philosophy, and is an author of Algorithms to Live By. He joins the show to explain how the same algorithms and data structures we use for our computer programs can be applied to the real world.

The post Algorithms to Live By with Brian Christian appeared first on Software Engineering Daily.

You have probably received a parking ticket that you felt was unfair,   but instead of fighting it, you paid the expensive price to get rid of it quickly. Fighting a parking ticket sounds like it would be so time consuming that it is a better decision to just pay for it. When Joshua Browder was faced with this situation, his response was different. He decided there should be an automated solution to fighting parking tickets, and he made the user interface a chat bot.

It’s not obvious why a chat bot interface for fighting parking tickets makes sense, but in my discussion with Joshua, he explained that a chat bot interface is actually useful for a wide variety of legal services. The way that a lawyer interacts with a client is often so mechanistic as to be similar to a robot.

This episode was a fascinating episode that serves as a great follow-up to the recent shows on chat bots–ChatOps, Bot Day, and Slack Bots. Thanks again to O’Reilly for giving me a ticket to Bot Day last month, as it really got me thinking about chat bots as an important user interface.

The post Robot Lawyer with Joshua Browder appeared first on Software Engineering Daily.