Tools for generating deep neural networks with efficient network architectures

In this episode of the Data Show, I spoke with Alex Wong, associate professor at the University of Waterloo, and co-founder of DarwinAI, a startup that uses AI to address foundational challenges with deep learning in the enterprise. As the use of machine learning and analytics become more widespread, we’re beginning to see tools that enable data scientists and data engineers to scale and tackle many more problems and maintain more systems. This includes automation tools for the many stages involved in data science, including data preparation, feature engineering, model selection, and hyperparameter tuning, as well as tools for data engineering and data operations.

Wong and his collaborators are building solutions for enterprises, including tools for generating efficient neural networks and for the performance analysis of networks deployed to edge devices.

Here are some highlights from our conversation:

Using AI to democratize deep learning

Having worked in machine learning and deep learning for more than a decade, both in academia as well as industry, it really became very evident to me that there’s a significant barrier to widespread adoption. One of the main things is that it is very difficult to design, build, and explain deep neural networks. I especially wanted to meet operational requirements. The process just involves way too much guesswork, trial and error, so it’s hard to build systems that work in real-world industrial systems.

One of the out-of-the-box moments we had—pretty much the only way we could actually do this—was to reinvent the way we think about building deep neural networks. Which is, can we actually leverage AI itself as a collaborative technology? Can we build something that works with people to design and build much better networks? And that led to the start of DarwinAI—our main vision is pretty much enabling deep learning for anyone, anywhere, anytime.

Generative synthesis

The general concept of generative synthesis is to find the best generative model that meets your particular operational requirements (which could be size, speed, accuracy, and so forth). So, the intuition behind that is that we treat it as a large constrained optimization problem where we try to identify the generative machine that will actually give you the highest performance. We have a unique way of having an interplay between a generator and an inquisitor where the generator will generate networks that the inquisitor probes and understands. Then it learns intuition about what makes a good network and what doesn’t.

Related resources:

Vitaly Gordon on “Building tools for enterprise data science”

“What machine learning means for software development”

“We need to build machine learning tools to augment machine learning engineers”

Tim Kraska on “How machine learning will accelerate data management systems”

“Building tools for the AI applications of tomorrow”