Software Engineering Daily

Machine learning frameworks like Torch and TensorFlow have made the job of a machine learning engineer much easier. But machine learning is still hard. Debugging a machine learning model is a slow, messy process.

A bug in a machine learning model does not always mean a complete failure. Your model could continue to deliver usable results even in the presence of a mistaken implementation. Perhaps you made a mistake when cleaning your data, leading to an incorrectly trained model.

It is a general rule in computer science that partial failures are harder to fix than complete failures. In this episode, Zayd Enam describes the different dimensions on which a machine learning model can develop an error. Zayd is a machine learning researcher at the Stanford AI Lab, so I also asked him about AI risk, job displacement, and academia versus industry.

Why ML is hard

The post Machine Learning is Hard with Zayd Enam appeared first on Software Engineering Daily.

Deep learning uses neural networks to identify patterns. Neural networks allow us to sequence “layers” of computing, with each layer using learning algorithms such as unsupervised learning, supervised learning, and reinforcement learning. Deep learning has taken off in the last few years, but it has been around for much longer.

Adam Gibson founded Skymind, the company behind Deeplearning4j. Deeplearning4j is a distributed deep learning library for Scala and Java. It integrates with Hadoop and Spark, and is specifically designed to run in business environments on distributed GPUs and CPUs. Adam joins the show today to discuss the history and future of deep learning.

The post Deep Learning with Adam Gibson appeared first on Software Engineering Daily.

Data science is typically done by engineers writing code in Python, R, or another scripting language. Lots of engineers know these languages, and their ecosystems have great library support. But these languages have some issues around deployment, reproducibility, and other areas. The programming language Golang presents an appealing alternative for data scientists.

Daniel Whitenack transitioned from doing most of his data science work in Python to writing code in Golang. In this episode, Daniel explains the workflow of a data scientist and discusses why Go is useful. We also talk about the blurry line between data science and data engineering, and how Pachyderm is useful for versioning and reproducibility. Daniel works at Pachyderm, and listeners who are more curious about it can check out the episode I did with Pachyderm founder Joe Doliner.

The post Go Data Science with Daniel Whitenack appeared first on Software Engineering Daily.

Translation is a classic problem in computer science. How do you translate a sentence from one human language into another? This seems like a problem that computers are well-suited to solve. Languages follow well-defined rules, we have lots of sample data to train our machine learning models.

And yet, the problem has not been solved–largely because languages don’t always follow rules. We have idioms and subtle contextual clues that make it hard to provide a computer with hard and fast rules for translation.

Unbabel is a company whose solution to translation puts a human in the loop to correct the error-prone translations that computers often make. In this episode, Vasco Pedro joins the show to explain Unbabel’s approach to translation, its technology stack, and the business applications for translation.

The post Translation with Vasco Pedro appeared first on Software Engineering Daily.

Medical imaging is used to understand what is going on inside the human body and prescribe treatment. With new image processing and machine learning techniques, the traditional medical imaging techniques such as CT scans can be enriched to get a more sophisticated diagnosis.

HeartFlow uses data from a standard CT scan to model a human heart and understand blockages of blood flow using simulations of fluid dynamics. In today’s episode, Razik Yousfi and Leo Grady from HeartFlow describe the data processing pipeline for the company and what their technology stack looks like.

The post Medical Machine Learning with Razik Yousfi and Leo Grady appeared first on Software Engineering Daily.

Data visualization tools are required to translate the findings of data scientists into charts, graphs, and pictures. Understanding how to utilize these tools and display data is necessary for a data scientist to communicate with people in other domains. In this episode, Srini Kadamati hosts a discussion with Jake VanderPlas about the Python ecosystem for data science and the different attempts at creating a data visualization library.

Jake VanderPlas is the Director of Research for Physical Sciences at the University of Washington’s eScience institute, where he also received his PhD in Astronomy. In addition to contributing to many Python data science libraries like scikit-learn, scipy, numpy, and matplotlib, he’s written multiple books that have been published by O’Reilly and has given many talks on data science tools and techniques. He’s also the co-creator of the Altair project, which is a declarative data visualization library for Python built on the Vega-Lite visualization grammar.

The post Python Data Visualization with Jake VanderPlas appeared first on Software Engineering Daily.

Data engineering is the software engineering that enables data scientists to work effectively. In today’s episode, we explore the different sides of data engineering–the data science algorithms that need to be processed and the implementation of software architectures that enable those algorithms to run smoothly.

The PANCAKE STACK is a 12-letter acronym that Chris Fregly gave to a collection of data engineering technologies including Presto, Cassandra, Kafka, Elastic Search, and Spark. In his current life, Chris travels around the world giving workshops on how to deploy and use the PANCAKE STACK. Before that, he was an engineer at Netflix, where he received an Emmy for Streaming Engineering Excellence.

The post PANCAKE STACK Data Engineering with Chris Fregly appeared first on Software Engineering Daily.

Scikit-learn is a set of machine learning tools in Python that provides easy-to-use interfaces for building predictive models. In a previous episode with Per Harald Borgen about Machine Learning For Sales, he illustrated how easy it is to get up and running and productive with scikit-learn, even if you are not a machine learning expert.

The post Scikit-learn with Andreas Mueller appeared first on Software Engineering Daily.

Machine learning can be used to generate music. In the case of Feynman Liang’s research project BachBot, the machine learning model is seeded with the music of famous composer Bach. The music that BachBot creates sounds remarkably similar to Bach, although it has been generated by an algorithm, not by a human.

BachBot is a research project on computational creativity. Feynman Liang created BachBot using Python machine learning tools to build a long-short term memory model. Our conversation explores artificial intelligence, music, and his approach to this research project.

The post Music Deep Learning with Feynman Liang appeared first on Software Engineering Daily.

You have probably read a news article that was written by a machine. When earnings reports come out, or a series of sports events like the Olympics occurs, there are so many small stories that need to be written that a news organization like the Associated Press would have to use all of its resources to write enough content to cover it all.

Wordsmith is a tool for automated content generation, and today’s guest Robbie Allen is the CEO of Automated Insights, the company that makes Wordsmith. He talks today about the wide range of uses for automated content, as well as how to engineer a product that takes data from a spreadsheet and turns it into a human-readable sentence.

Robbie is also speaking at the O’Reilly Artificial Intelligence Conference in New York, September 26-27.

The post Automated Content with Robbie Allen appeared first on Software Engineering Daily.