Ali Ghodsi, CEO & CO-Founder at Databricks, has been a major influence in three big open source projects – Apache Mesos, Apache Spark and Apace Hadoop. What’s interesting about all of these three projects is that there are very successful companies around these projects. We sat down with Ghodsi to talk about these Open Source projects and machine learning.

According to Ghodsi, machine learning replaces manual, repeatable processes. Older systems were rule based which would bring some level of automation, but they had their own limitation. First of all, any rules-based systems can be easily gamed. Second, you can’t really use such a system to monitor and respond in situations where billions of people are interacting with each other in real time.

A chat program, which is used by teenagers can be a great example. You may want to monitor it for foul language or any alarming content; it can’t be done with rules and teenagers know how to bypass things. “Machine learning overcomes that problem and becomes a very powerful tool in such use-cases,” said Ghodsi.

Watch on YouTube: https://youtu.be/XSuJgC5DwQc