The Frontside Podcast

Kevin Nater, CEO & Co-Founder |

Show Notes:

Resources:

Transcript:

Coming Soon!

Katharine Beaumont: @katharinecodes

Show Notes:

In this episode, we hit the topic of machine learning from a 101 perspective: what it is, why it is important for us to know about it, and what it can be used for.

Transcript:

CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 94. My name is Charles Lowell, a developer here at The Frontside and your podcast host-in-training. Today I’m going to be flying it alone, but that’s okay because we have a fantastic guest who’s going to talk about a subject that I’ve been dying to learn about. But you know, given the number of things in the world, I haven’t had a chance to get around it. But with us today is Katharine Beaumont who is a machine learning consultant. And she’s going to talk to us, not surprisingly, about machine learning. So welcome, Katharine.

KATHARINE: Hello. Thank you very much for having me.

CHARLES: No, no, it’s our pleasure. So, I guess my first question is, because I’m very much approaching this from first principles here, is what is machine learning as a discipline and how does it fit into the greater picture of technology?

KATHARINE: Okay. Well, if you think about artificial intelligence which is one of those slightly undefinable fields because it encompasses so much, so it encompasses elements of robotics, linguistics, math, probability, philosophy, it has six main elements. So, a really basic definition of machine learning is getting, and this comes from Arthur Samuel in 1959, it’s about getting computers to learn without being explicitly programmed. And that’s hugely paraphrasing. But machine learning is an element that sits under the wider discipline of artificial intelligence. Artificial intelligence is one of those tricky to define fields because people have different opinions about what it is. And obviously philosophers can’t agree what intelligence is, which makes it slightly complicated.

But artificial intelligence as a broad brush is a discipline that borrows from philosophy, math, probability, statistics, linguistics, robotics, and spawned subfields like natural language processing, knowledge representation, automated reasoning, computer vision robotics, and machine learning. Machine learning is the, in a sense, the mathematical component of artificial intelligence in that from a basic point of view, even though you’re looking at it from the perspective of computer science, you’re utilizing algorithms that a lot of mathematicians will say, “Look, we’ve been doing this for years. And you’ve just stolen that from us,” that try and find patterns in data. And that pattern could be as basic as mapping, say, the square footage of a house to the price that it will sell at and making a prediction based on that for future examples, or it could be looking for patterns in images.

CHARLES: Okay. You mentioned something that I love to do. I love stealing ideas from other disciplines. It feels great.

KATHARINE: Who doesn’t?

CHARLES: Yeah. It’s like free stuff. And the best part of ideas is the person who had it still has it after you’ve lifted it off of them.

KATHARINE: Yeah. You just have to reference and then it’s not plagiarizing.

CHARLES: Yeah. So, how did you actually get into this?

KATHARINE: Well, a few years ago, I was desperately bored in my job.

CHARLES: So, what was that job that you were working on that was so desperately boring? You don’t have to name a company.

KATHARINE: Oh, I won’t name the company but I will – I have to make a confession now which links back to something that we were saying off recording earlier, which was that it was doing web development. So, I’m sorry. And that’s not to say that web development is boring at all. It’s just that I wasn’t particularly engaged, which is not a reflection on web development.

CHARLES: No, no, no. I actually came – I was doing, before I got into web development, I was actually doing backend stuff for years. That was all I did.

KATHARINE: Yeah, me too. I would have described myself as a server-side Java developer who then cross-trained into Ruby. And I thought I’d be doing exciting backend things in Ruby. But unfortunately, it was more, “We’d like you to move this component from this part of the page to this part of the page.” And I didn’t really connect with that. And I started to wonder if I even should be a developer.

CHARLES: Wow.

KATHARINE: Larger forces than myself were at work to try and push me into management or analysis. And as happens, I think, after a few years. So, I started doing, in my spare time, looking at a website (and I’m sure you’ve heard of it) Coursera.

CHARLES: Yeah.

KATHARINE: So, this is the birthplace of the massive online, I can’t remember what the second O is, MOOCs. Massive Online something learning. Maybe a Q in there. I’m not sure what. Do you know what the acronym is?

CHARLES: I actually don’t know.

KATHARINE: Well, MOOCs anyway. Massive online learning courses. And there was one offered Andrew Ng from Stanford on machine learning. So, I took that and I just loved it. I really enjoyed it. And I really connected with the programming. I really enjoyed the programming. It was very fulfilling. So, it grew from that, really. And now, I’ve decided to go back to university. So, I’m a mature postgraduate student and I’m just currently weighing up my PhD options. So, whether to sacrifice four years for the greater good and the pursuit of knowledge or go back into an employment. So, we’ll see. We’ll see. And I’m quite enjoying not being employed, I have to admit. Or being employed on a freelance basis. It’s wonderful.

CHARLES: Right, right, right. Now, a couple of things stuck me when you were talking about – so obviously, you’re studying a lot of the mathematics behind it. And you said that machine learning involves a lot of the – it’s the mathematical component of artificial intelligence. But what strikes me is learning, to me, implies a lot of statefulness where you’re accumulating state. Whereas my experience with mathematics is usually you're solving equations. You start from some set of facts and whether it’s a dataset or some other thing, and you derive, boom, boom, boom, boom, boom, you get your answer. Whereas with learning, at least when I think about school learning, like spelling or, I don’t know, paleontology or something, you’re accumulating facts over a very long time. And the inferences that you make are not necessarily – they’re drawn from all fo the sources that you got over all that period of time rather than some one set of facts that then you make this logical argument and poof, presto, you’ve got your answer. How does that square? I guess it’s just a little bit off from my experience with mathematics.

KATHARINE: So, I am being a bit reductionist. So probably, one way to explain it is that essentially, behind a lot of the machine learning algorithms, you’re inputting numbers. And that might be the percentage of red, green, blue in a pixel for example. Or it might be the diameter of a wheel, for example, if you’re looking at a component of a car. Or it might be a binary configuration if you want to input the configuration of a control panel, for example, and you’re looking for anomalies. And you’re running these numbers through an algorithm. And what you’re getting out is either a continuous value, if you’re looking at a problem with continuous data like house prices, or you’re getting a probabilistic output like 60% certain these pixels together make a cat, for example. So, I am simplifying by saying it’s math because what you're really doing is looking for patterns in data but a way to get a computer to understand it is to somehow input it as numbers, essentially, and to get numbers out of it.

CHARLES: Oh.

KATHARINE: Yeah, it’s more algorithms, really. And I shouldn’t have said that it was essentially math because I’m sure I’m going to get shouted at on the internet.

CHARLES: Well, I certainly don’t want to get you in trouble. But maybe that’s a point that we should shy away a little bit, the high theoretical stuff, and bring it back. If I’m excited, not even if I’m excited, why should I be excited about it? I’ve heard that it’s a hot topic. I’ve heard that a lot of people are excited about it. Is there a way that I as someone who has no specialization in this might actually be able to bring some of these techniques to bear on the problems that I’m working on? Perhaps even without understanding them first, like understanding how it works. What are some problems that I might be able to attack with these techniques?

KATHARINE: Yeah, absolutely. So actually, and one of the things about machine learning that I should say is don’t think, “Oh, it’s not for me. I’m rubbish at math. I don’t understand these concepts. I’m not willing to get my head around an algorithm,” because there are so many pre-configured APIs available from big companies like Google and Amazon and Microsoft and IBM, and I’m sure many, many more. And I’m not paid by any of them, I should say. So, you don’t need to understand the inner workings of an algorithm to use it.

So, one example is speech-to-text. So, if you imagine that you’re working on a website and you want to make it accessible, maybe you could have a component to your navigation bar that allows users to record their voice and say, “I want to navigate to the shopping cart,” for example. And machine learning would be behind that processing. So maybe, behind it you’d have an API, I’ve used a few of them before just to play it, where you make a call to, say, IBM service and it returns you the text. And in your program you match on keywords like shopping cart and then change the menu bar for them. So, that’s one really simple way you could do it.

Another more complicated way to do it is to implement something like a recommender system. So, say you have a website where you offer customers products of some description. And the most famous example of this is Amazon and Netflix. Amazon, the shopping site, rather than now the big, big corporation. And you see what other customers like you bought, or Netflix, what you might enjoy. And that’s based on taking your information, comparing your viewing habits to other people’s viewing habits, and then drawing some kind of correlation between the programs you watch and trying to find programs that other people have watched that you haven’t, that you might enjoy. That’s more complicated, to be honest.

CHARLES: But that is an example. Machine learning is what underlies all that.

KATHARINE: Absolutely. And at the heart of some recommender systems, the mathematics behind it is finding a way to quantify people’s preferences and measuring distances between them. But you don’t need to understand that to understand the basics of how a recommender system works.

CHARLES: Okay. And so, how does a recommender system work?

KATHARINE: So, imagine me, yourself, and Mandy each read four books. And we rate them. But I read four books, you read three of the same books, and one different one, and Mandy reads three of the same books as me and one different one, for example. So, we’ve got a little gap but we’re not really sure what the other person will think. And we know the genres of the books. And you can compare the genres and the ratings. So, you might rate sci-fi 6 out of 10 and romance 7 out of 10. And I might rate sci-fi and romance in equal ways. So then you might say, okay, there’s a similarity between our preferences. So for this book, that Charles read, Katharine might like it.

CHARLES: That makes so much sense.

KATHARINE: Yeah. And maybe Mandy only likes romance, only rates it 0.3 for example. So we think, “Okay, well Mandy might not be able to recommend a book to Katharine and Charles.”

CHARLES: Right, I see. Implicit in this though is there’s this step of the actual learning, I guess. Or the actual teaching. How do you actually teach? Again, and this is kind of me trying to wrap my head around the concept, is I’ve got these set of facts and I’m inferring and I’m pattern-matching and I’m trying to draw conclusions with some certainty from this set of data. But is there this distinct actual teaching phase where you have to actually teach the computer and then it takes new facts and gives stuff? How does that work? How does it incorporate the different – I guess what I’m saying is, are there distinct phases? Or is it…

KATHARINE: Yes, and it’s not the same for every algorithm. So, I’m going to try and give you two examples of training. So, there’s something called online learning where as a new example comes in, for example it might be – I’ll just explain what a classifier is. So, this is a brief diversion. So, a classifier is a machine learning process where you’re trying to put information in and you’re trying to get discrete information out. So, discrete meaning like it’s a cat or it’s a dog or a weasel or a minion or something like that. Or, it’s cancer or it’s not cancer. Whereas continuous output might be the price of a car. So, in a classifier you’re trying to work out what type something is. So, a really good example is there’s a Google project where they’ve clustered artworks. So, they’ve taken lots of different artwork and their algorithms, which I won’t explain now, have determined, “This is a ballet dancer. So, we’re going to group all of these ballet dancer paintings together. This is a landscape, so we’re going to group all the landscapes together.” So, online learning, you might get a bit if information in, like a picture, and you will classify it and you add it to your existing information. Whereas another type of learning is you take all of the information you have, you train the algorithm, and then you make predictions. So, you either make predictions as you go along with online learning, or you do all of the work upfront. So, one algorithm – have you heard of decision trees?

CHARLES: No, I haven’t.

KATHARINE: So, do you ever read those rubbish teen magazines where you have a flowchart and it starts at the top like, “Do you like cats? Yes or no?”

CHARLES: Oh right, yeah.

KATHARINE: “Do you likes dogs? Yes or no,” and it tells you what kind of a person you are or what make up you should wear or something like that.

CHARLES: Right, right, right, yeah.

KATHARINE: Yeah, so decision trees are kind of like that. One example is you might get information about, it’s a famous toy dataset, information about passengers on the Titanic about gender and age. And we all know the techniques on the Titanic, women and children first. I’ve lost my use of normal English. I’m sorry.

CHARLES: Right. Maybe like a trope?

KATHARINE: Yeah. So, women and children first. So, you put this data in a decision tree. And what happens is at the beginning you have all of this data. So, person A is a male. They’re in their 50s. This is the type of ticket they had. And this is their income. I don’t think income is one of them, but just as an example. And then you have person B, person C. So, you might have a hundred people. And the decision tree algorithm goes, “Okay, if I just looked at one of these features like gender, would that differentiate the people the most?” So, it already has the answer as to whether or not they survived or did not survive. And it’s looking for the one feature that gives it the most information. And then it will split on that feature. So, you go form your thing at the top and the first question might be, “Were they male or female?” And then the decision tree will split down a level. And then your algorithm will go, “Alright, what’s the next feature?” Maybe the next feature is, “Were they under the age of 30?” for example. And it works down. And you end up with this sort of flowchart. And once it’s trained, you then get a new example you have, person X. and you basically just work your way through the decision tree to make the prediction of whether they survived or did not survive.

CHARLES: I see. And so, do you do it with some sort of certainty? Because there’s going to be variation, right? You’re going to have some people who are a poor fellow in his 70s who survived. Like, it’s not certain that he went down but there’s some probability at the end?

KATHARINE: Yes. Because you’re using it to make a prediction, there is always a probabilistic element. And it depends on the decision tree algorithm. So, there are some decision tree algorithms that really don’t work with contradictory data, for example.

CHARLES: I see.

KATHARINE: There’s an element of picking your algorithm. If you’re approaching a machine learning problem, you have three elements. The first one is choosing your feature and your algorithm. Then you have evaluating it, so you need a way of saying how good or bad the algorithm’s doing on your data, how accurate is it for example. And then you have optimizing it, which is the dark art of machine learning.

CHARLES: Right. That’s the wrap across the knuckles. It’s coming up with wrong numbers.

KATHARINE: Yeah. That’s – oh no, this took two weeks to run and I need it to take 20 seconds. Or, this is only 60% accurate and I need it to be more accurate. And it’s easy, just as an example I’m just working on a course that I’m giving in a few weeks’ time. And I just took a day to set up a website called Kaggle, K-A-G-G-L-E, for wine quality. So, it’s got acidity, citric acid, residual sugar, chlorides. They’re the features. They're the components that you’re going to put in. And it has a quality score. So, what you’re trying to do is you’re trying to find a relationship between the features and the quality. And it’s very easy to get it to work. So, I now have it working. But I have it working with a really low accuracy. So, it took maybe five minutes to get it to work and it’s going to take me about half an hour to make it more accurate, and that’s the optimization element.

CHARLES: I see. How stable are these processes? Is it finicky and fragile so that if you get new types of features it just throws things way off? Or are there ways you could control for that?

KATHARINE: Well, that’s a particular type of problem and that all links in with the evaluation and optimization. And that’s to do with something called overfitting. So, decision trees is an algorithm notorious for overfitting, depending on the data, that is. So, overfitting is when you get the algorithm to perform really well on the training data. And then you feed it in a new example and it might grossly misclassify it because it hasn’t learned to generalize beyond the examples that you’ve given it.

CHARLES: I see. Okay. So, it’s just too concrete. It hasn’t recognized deep patterns. It’s only recognized something superficial.

KATHARINE: Yes. So typically, if you have a finite amount of data, you only train your algorithm on a certain percentage. And then you test it on the rest.

CHARLES: I see.

KATHARINE: So, you hold back some data. But back to our conversation earlier about when does the training happen? Another example is something called K-nearest neighbors. You could just imagine that means three nearest neighbors, for example. So, we’re trying to find who we’re most similar to in a room. So, it’s a room full of people. And all the people are standing next to already similar people, for example. So, you might have a room where marketing’s in one corner and the software developers are in another corner and the project managers are in another corner. And you go it and you’re looking for the three people who are the most similar to you and you’re going to go and stand in that group, for example. So, in that type of machine learning, the training is happening as each new sample comes in, rather than upfront. And there are drawbacks to both methods and there are positives to both methods. And really, in a horrible, unsexy way, it’s to do with the data. And that’s normally where most people because it’s the most boring part when you’re talking about machine learning.

CHARLES: Is the data?

KATHARINE: Yeah. It’s this backlash from data scientists, from the golden age of data scientists where it was the hottest job on the internet to now everyone cringing going, “Oh, I really don’t want to deal with my data.” But with any machine learning problem, you can’t just go, “Okay. Here you go, Charles. Here’s a dataset. Learn something from it.” You need context. You need to understand it. You need to have an idea of what you’re looking for. So, you’re getting the machine to learn but you're using it as a tool to complement your knowledge, really. And you’re feeding in your knowledge to it. And part of that are the decisions that you make on the algorithms to use.

CHARLES: Okay.

KATHARINE: And what you’re looking for.

CHARLES: So, here’s something that I’m wondering is related, and again I have no idea – for some reason I always associate when people talk about neural nets as being related to teaching a computer something. Is that part of the discipline of machine learning? Or no?

KATHARINE: I would say it is. But it has its own cool and trendy title of Deep Learning. But it’s very much powerful machine learning. So, let’s go back to this. This is a classic example of machine learning. It’s probably the first example you’ll come across if you do any course. House prices. So, imagine that you have a piece of paper and you're going to draw one line at one side and that is the price of a house, and you’re going to draw a line at the bottom which is the size in square feet, and you’re going to plot examples that you have. And you might find that there’s a linear correlation between the two. So, you’ll draw a line and that line of best fit is the human equivalent of doing linear regression on a computer, for example, where you’re just trying to find a linear correlation between two things.

And a lot of the principles in linear regression, which is a very simple learning algorithm, are found in some examples of neural networks, so some basic examples of neural networks. But instead of having one input, the size in square feet, you might have 20. And you might be repeating that process of trying to find the line of best fit with different combinations of features in different places again and again. And it scales up in complexity very quickly. But it’s very similar basic principles. I’m hesitating to say ‘very similar’ because they’re notoriously more complicated.

CHARLES: Yeah. I guess I’m not really divining what exactly, what makes it – why is it called a neural net? What makes it special? It sounds like if I’m just comparing the regressions of house prices, I’m comparing those datasets over and over again, how is that different from just a loop? What are you getting out of it?

KATHARINE: Historically, neural networks come from a very simplified idea about how the brain works. So, in the early 20th century people had performed autopsies and divined the inner workings of kidneys and hearts and livers. And the brain was still a bit of a mystery. And then 2 men jointly won the Nobel Prize for Physiology, and I’m going to pronounce these names wrong, so I’m sorry. I think it’s Santiago Ramón y Cajal is one and Camillo Golgi is another. And they completely disagreed about the brain but they used a staining technique from Golgi to look, using silver nitrate, at the cells in the brain. And the idea of the neuron doctrine was borne out of that, that the simplest unit to look at the brain at in order to understand it is the level of the neuron, this cell in the brain.

And from there, several – well, everybody was a polyglot really, back then, so I don’t want to say computer scientists. So, you had people like Frank Rosenblatt with perceptrons, Marvin Minsky and so many other people looking at a very simple idea which is that a neuron either fires or doesn’t. And then you’re linking boolean algebra to this cell. So, you’re saying it either fires or it doesn’t. And from that principle, people started drawing similarities between neurons and a basic function machine.

And when I say function machine, I mean imagine when you were in elementary school and you’re learning how adding up works. You might have a box with a plus on it and your teacher says, “I want you to put a three in a box and a four in a box and I want you to add them together. And what do you get out?” And obviously the answer is seven. And you can think about that little box with a plus as a function machine. So, now you could think of a little mathematical function machine where you put in some inputs and there something happens in the box. And then you’ll either get a one or a zero out of it.

CHARLES: And so, that’s like your neuron, is the little box?

KATHARINE: Yes.

CHARLES: Okay.

KATHARINE: Yes. So since then, the neuron doctrine is pretty much contested. There are several other elements of the brain that compose thinking and the circuitry. So, any cognitive scientist listening to this will say, “That’s really not how the brain works.” You have to say it with a lot of disclaimers. But the whole idea of neural networks was borne out of this idea of thinking of a neuron as like a function machine.

CHARLES: Right. And also, it doesn’t actually discount the usefulness of neural networks. There are a lot of things where people didn’t find what they set out to find but what they found was useful.

KATHARINE: Absolutely. Yeah, and they are incredibly powerful, especially with multilayer networks which are deep networks or deep learning.

CHARLES: Okay. So, I didn’t want derail you from your explanation. So, you’ve got these little function boxes and those are the kind of neurons inside the neural network?

KATHARINE: Yes. So, what happens is you might have a layer of 10 of them and you might have another layer after that of another 10. And each of them are connected in a simple neural network. And then you might have an output layer of five because you’re looking to classify, I don’t know, an apple into five different types of apple, for example.

CHARLES: Now, when you’re talking about a later, you’re talking about, I’ve got the outputs of one layer of the network are the inputs to the next layer.

KATHARINE: Yes. And in different neural network architectures they’ll be connected differently. But in a simple neural network, you can assume that every neuron is connected to every neuron in the next layer. So, if you have two 10-layer, two layers each with 10 neurons in, the top neuron in one layer is going to have 10 connections going out of it into the next layer.

CHARLES: Oh really? Wow, that’s interesting.

KATHARINE: Yes. And each connection has its own sort of configuration.

CHARLES: So, you’re like cross-wiring all the – okay. Wow, that’s kind of…

KATHARINE: And yeah, that’s where the complexity comes in.

CHARLES: Yeah. I was thinking it was like a simple exponential fan-out. But it’s even more complicated, the number of combinations you can get.

KATHARINE: Yes. But in theory, it’s very simple because each neuron is like this function machine with inputs coming in and something going out. It just might go out to several different locations. But it scales up in complexity very quickly. So say we’re classifying apples. So, we have five different attributes of an apple like its color, its texture, its weight, acidity. I don’t know how else you would measure an apple. How shiny it is, for example. And we’re putting all of that information in. And each one of those things that I’ve just listed is a feature. So, we might have an input per feature and we’ll link them all up to the neurons in a layer and we’ll move that information onto the next layer. And the really important thing is what’s happening on those connections between them. Because on the connections you have weights, which is a way of changing the input from one neuron to another. So, the weight might be like 0.5 for example. So, it squashes the input. Or it might magnify it. And then you get the output.

Now, once you have the output you can compare the output with what you know the right answer is. And then you have this idea of an error. So, you might be like, “You got this so wrong. It’s not a Braeburn apple at all. It’s actually a Granny Smith apple. And what you do is with each example that you train in, you use your information about the error to train the network. Because what you’re trying to do is get the error to be as small as possible. And one of the techniques of that is called back propagation. And it’s notoriously difficult to understand because it involves partial derivatives and a large element of calculus. But essentially, what you’re doing is comparing the right answer with the answer that the network gave it and asking it to go back and change it, change those connection weights.

CHARLES: And so, do you make them fluctuate at random or is there some – is there a method to the madness of changing the weights?

KATHARINE: There is a method to the madness and it’s called back propagation. And the reason I linked linear regression in early, so our really simple map of house size in square feet, and the price, is because it uses a similar technique called gradient descent which is an algorithm for looking at the error and changing the weight, those numbers on the connections, to try and get it to a minimal point. So, if we go back to our house price problem, I just want you to imagine in your mind that we’ve got this one axis going up which is the price, and one going across which is the size in square feet. And you’ve got line drawn, a diagonal line. If you just imagine now in your mind moving that line down till it’s completely flat at the bottom, and then moving it up so it’s vertical, so it’s aligned with either axis in every point in between, what you could do is you could take the error on each of those lines.

So, if we imagine we have these two axes, we have the price of a house on one side and we have the size in square feet in another. And we’re going to draw a line from the top axis and we’re going to sweep it down and draw a line at each point as it goes down until it’s aligned with the bottom axis. And at each point we draw that line, we measure the error. So, what we’d do for that is all of the little points, all of the example data, we’d measure the difference between them and the line and we’re going to, say, add them all up. So, what you’d end up with is you’d end up with a graph mapping the error against the gradient at all fo those different points. And it would look like a bowl. You’d have a lowest point. You’d have a point for some gradient where the error is the lowest.

CHARLES: Right, yeah. Okay. I’m seeing it. I think I’m seeing it. So, you want to take that error function and you want to, what is it? Now this is – boy, I’m going back to high school math. You would take the derivative and find out where the tangent is and that’s your min point? That’s the root of the equation and that’s the point where your error is lowest?

KATHARINE: Yeah, exactly. Or there’s an algorithm called gradient descent that does it automatically by taking little steps. So, it looks at the tangent of the gradient at a point and says, “If I move the gradient down is the error going to decrease. And if so, move in that direction.” So automatically, it tries to take steps to get to the bottom. And optimization problem with that is that you can configure the step size. So, you could take tiny, tiny steps and take forever to get to the bottom or you could take massive steps and completely miss the bottom. So, you can imagine it like walking down a hill. If you’re a minion then it will take a really long time because you’re tiny. And if you’re a giant, you might never get to the valley.

CHARLES: Right. You might just leap right across the chasm.

KATHARINE: Yeah, just miss it completely. So, that’s linear regression where we’re looking. And that’s really, even though we have a two-dimensional graph that’s a one-dimensional problem because we just have the one input feature. Now, when we’re looking at neural networks and we’re looking at gradient descent in neural networks, each one of those connections is something that we’re trying to configure to reduce the error. So suddenly, you have maybe a hundred-dimension landscape and you’re trying to get to the bottom of a hill. And there might be several local optimas and one deepest valley, but you might have lots of other valleys that you could get stuck in. So, it becomes a very difficult problem. Does that make sense?

CHARLES: Yes. No, that does make sense. I’m just trying to let it sink in.

KATHARINE: It’s completely impossible to visualize a hundred dimensions, yeah.

CHARLES: I actually had to sit back and kind of close my eyes and stare up at the ceiling.

KATHARINE: I think the trick is not to think about it. I heard someone say, there’s another fantastic course on Coursera by a famous computer scientist studying neural networks called Geoffrey Hinton. And his advice in one of the videos on visualizing these multidimensional landscapes, say it’s 15 dimensions, is to close your eyes and shout, “15!” And that hasn’t worked for me, but I’m sure it’s worked for some people.

CHARLES: But it certainly probably makes you feel better.

KATHARINE: Yeah. I think it’s one of those things that’s just beyond comprehension. But we can just quietly accept it.

CHARLES: Right. It’s just – yeah, what’s nice about I guess math is you just don’t have to understand it. I mean, you do. You just understand that there really is no mapping to our physical experience. And that’s okay. And just let that go. It’s just like, this is just some…

KATHARINE: Yeah.

CHARLES: We had some numbers that existed in the domain of understanding, that we can understand. And there are just some rules that we follow and if you look at the intermediate steps, well the points don’t really exist in that domain of physical experience and understanding. And that’s okay. We just accept and let it go. We just hope that at some point, we can translate that model back into the domain of ‘we can understand it’.

KATHARINE: Oh, completely. There’s a lot of faith.

CHARLES: Yeah.

KATHARINE: And also, I remember coming into machine learning and thinking, “This is like magic. It’s amazing.” And then you study it a bit and you’re like, “This is so easy. This is just basic – this is just functions. These are just glorified function machines.” And then you look into it some more and you’re like, “Nope. It’s definitely magic.”

CHARLES: Yeah. It’s a phase of, every time you come up against the wall, right? And then you realize, “Oh no, it’s actually something that I can close my mind over,” until you come to the next hurdle of magic.

KATHARINE: Yeah. You think you’ve got a grasp on things and you think, “I know the landscape,” and then you suddenly realize how much more there is to learn. And that sinking feeling that you’ll never learn it all.

CHARLES: Yeah, yeah. Yup. Unfortunately, it seems like in tech that’s like, that’s just the condition.

KATHARINE: Yeah, like all of my sad, unread books.

CHARLES: If I wanted to get into just really start experimenting with this stuff and start saying, “Maybe I can utilize some of these techniques for some of these problems that I’m encountering,” Where would be a good place to get started? What libraries? What online resources? What people are good to follow and ask questions of?

KATHARINE: Okay. Let’s start with websites. So for a start, this website called Kaggle which I mentioned earlier, and that is K-A-G-G-L-E dot com. And that has a lot of dataset resources. It also has a community of people discussing how they use the datasets. It has competitions. And it has a lot of links. I discovered recently as well a really good blog. It’s on Medium and it is called ‘Machine Learning for Humans’. And that’s really well-written. I really like it, actually, and it has a good section on resources called ‘The Best Machine Learning Resources’. And I should probably plug my own blog, but this one’s so much better.

CHARLES: But please do.

KATHARINE: No, no. I have to actually write stuff for it. But there’s a lot of things there about, well, if you want to learn linear algebra, what if you want to learn probability and statistics, calculus, and then just go straight to machine learning and pick up the math on the way. I would say go on Coursera, because there are courses like Andrew Ng’s course on machine learning from Stanford. And Geoffrey Hinton from the University of Toronto. But there are also courses there on things like calculus and probability and statistics if you want to level up your math. If you don’t want to do anything to do with the math, I would say go to the vendor websites, like AWS, Google. If you’re into Java, go on deep learning for J. And a lot of them have tutorials that complement their products. Deep learning for J is one of my favorites at the moment. It’s an open source Java library. It’s pretty plug and play, actually. You don’t need to understand a lot of it to get started with it. But it helps. And obviously then there’s TensorFlow although personally, I find just other Python libraries like SciPy a lot easier than using TensorFlow.

And I think naturally you'll find the resources and the people to follow on Twitter from that. But the crucial thing I’d say is don’t get hung up on which language to start playing around with. So, a lot of people say, “Oh, I must need to know Python. I must need to know math.” And really, you don’t. It just depends on what level you want to approach things at. So, if you want to write your own gradient descent algorithms, then Python is probably more for you, or Matlab or R or something like that. But there are libraries where you can do it in Java. I’ve heard rumors that there’s a JavaScript library and I wouldn’t be surprised. So, I would just have a look at what’s out there. But try and get a grasp of the fundamentals just from an intuition point of view, because it will make your life so much easier. You might for example realize that you're using the completely wrong algorithm for the problem that you're looking at. And that’s invaluable.

CHARLES: Yeah. Knowing what not to do certainly is. Alright. Well, thank you so much for that, Katharine. Thank you for being on the show. Thank you for curing us of at least a small portion of our ignorance. And if people want to get in touch with you perhaps and continue the conversation, or follow you, what’s a good place to get in touch?

KATHARINE: Sure. Probably tweet me on Twitter. I’m @KatharineCodes but it’s spelled like Katharine Hepburn. It’s K-A-T-H-A-R-I-N-E codes. Because when I joined Twitter, I didn’t have much of an imagination. I still don’t. So, it’s not particularly clever. But it’s there.

CHARLES: Alright. Well, fantastic. And for everybody listening at home, you can also get in touch with us. We’re @TheFrontside on Twitter. Or you can just drop us a line at [email protected]. Thanks for listening and we will see you all next time.

Julie Moronuki: @argumatronic | argumatronic.com

Show Notes:

This episode is a follow-up episode to the one we did with Julie in September: Learn Haskell, Think Less. We talk a whole lot about monoids, and learning programming languages untraditionally.

Transcript:

CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 93. My name is Charles Lowell, a developer here at The Frontside and I am your podcast host-in-training. With me today from The Frontside is Elrick also. Hello, Elrick.

ELRICK: Hey.

CHARLES: How are you doing?

ELRICK: I’m doing great.

CHARLES: Alright. Are you ready?

ELRICK: Oh yeah, I’m excited.

CHARLES: You ready to do some podcasting? Alright. Because we actually have a repeat guest on today. It was a very popular episode from last year. We have with us the author of ‘Learning Haskell: From First Principles’ and a book that is coming out but is not out yet but one that we’re eagerly looking forward to, Julie Moronuki. Welcome.

JULIE: Hi. It’s great to be back.

CHARLES: What was it about, was it last October?

JULIE: I think it was right before I went to London to Haskell [inaudible].

CHARLES: Yeah.

JULIE: Which was in early October. So yeah…

CHARLES: Okay.

JULIE: Late or early October, somewhere in there.

CHARLES: Okay. You went to Haskell eXchange. You gave a talk on Monoids. What have you been up to since then?

JULIE: Oh wow. It’s been a really busy time. I moved to Atlanta and so I’ve had all this stuff going on. And so, I was telling a friend last night “I’m going to be on this podcast tomorrow and I don’t think I have anything to talk about.”

[Laughter]

JULIE: Because I feel like everything has just been like, all my energy has been sucked up with the move and stuff. But I guess…

CHARLES: Is it true that everybody calls it ‘Fatlanta’ there?

JULIE: Yeah. [Laughs]

CHARLES: I’ve heard the term. But do people actually be like “Yes, I’m from Fatlanta.”

JULIE: I’ve heard it a couple of times.

CHARLES: Okay.

JULIE: Maybe it’s mostly outsiders. I’m not sure.

CHARLES: [Chuckles]

JULIE: But yeah, it’s a real cool city and I’m real happy to be here. But yeah, I did go in October. I went to London and I spoke at Haskell eXchange which was really amazing. It was a great experience and I hope to be able to go back. I got to meet Simon Payton Jones which was incredible. Yeah, and I gave a talk on monoids, monoids and semirings. And…

CHARLES: Ooh, a semiring.

JULIE: Semiring. So, a semiring is a structure where there’s two monoids. So, both of them have an identity element. And the identity element of one of them is an annihilator. Isn’t that a great word? It’s an annihilator…

CHARLES: Whoa.

JULIE: Of the other. So, if you think of addition and multiplication, the identity element for addition is zero, right? But if you multiply times zero, you’re always going to get to zero, so it’s the annihilator of multiplication.

CHARLES: Whoa. I think my mind is like annihilated.

[Laughter]

JULIE: So, it’s a structure where you’re got two monoids and one of them distributes over the other, the distributive property of addition and multiplication. And the identity of one of them is the annihilator of the other. Anyway, but yeah, I gave a history of where monoids come from and that was really fun.

CHARLES: Yeah. I would actually like to get a summary of that, because I think since we last talked, I’ve been getting a little bit deeper and deeper into these formal type classes. I’m still not doing Haskell day-to-day but I’ve been importing these ideas into just plain vanilla JavaScript. And it turns out, it’s actually a pretty straightforward thing to do. There’s definitely nothing stopping these things from existing in JavaScript. It’s just, I think people find type class programming can be a tough hill to climb or something like that, or find it intimidating.

JULIE: Yeah.

CHARLES: But I think it’s actually quite powerful. And I think one of the things that I’m coming to realize is that these are well-worn pathways for composing things.

JULIE: Right.

CHARLES: So, what you encounter in the wild is people generating these one-off ways of composing things. And so, for a shop like ours, we did a lot of Ruby on Rails, a lot of Ember, and both of those frameworks have very strong philosophical underpinnings that’s like “You shouldn’t be reinventing the wheel if you don’t have to.” I think that all of these patterns even though they have crazy quixotic esoteric names, they are the wheels, the gold standard of wheel. [Laughs] They’re like…

JULIE: Right.

CHARLES: We should not be reinventing. And so, that’s what I’m coming to realize, is I’m into this. And last time you were talking, you were saying “I find monoids so fascinating.” I think it took a little bit while to seep in. But now, I feel like it’s like when you look at one of those stereo vision things, like I’m seeing monoids everywhere. It’s like sometimes they won’t leave me alone.

JULIE: In ‘Real World Haskell’ there’s a line I’ve always liked. And I’m going to misquote it slightly but paraphrasing at least. “Monoids are ubiquitous in programming. It’s just in Haskell we have the ability to just talk about them as monoids.”

CHARLES: Yeah, yeah.

JULIE: Because we have a name and we have a framework for gathering all these similar things together.

CHARLES: Right. And it helps you. I feel like it helps you because if you understand the mechanics of a monoid, you can then when you encounter a new one, you’re 90% there.

JULIE: Right.

CHARLES: Instead of having to learn the whole thing from scratch.

JULIE: Right. And as you see them over and over again, you develop a kind of intuition for when something is monoidal or something looks like a semiring. And so, you get a certain intuition where you think, “Oh, this thing is like a… this is a monad.” And so, what do I know about monads? All of a sudden, this new situation like all these things that I know about monads, I can apply to this new situation. And so, you gain some intuition for novel situations just by being able to relate them to things you already do know.

CHARLES: Exactly. I want to pause here for people. The other thing that I think I’ve come in the last three months to embrace is just embrace the terminology.

JULIE: Yeah.

CHARLES: You got to just get over it.

JULIE: [Chuckles]

CHARLES: Think about it like learning a foreign language. The example I give is like tasku is the Finnish word for pocket.

JULIE: Right.

CHARLES: It sounds weird, right? Tasku. But if you say it 10 times and you think “Pocket, pocket, pocket, pocket, pocket.”

JULIE: Yes, yeah. [Laughs]

CHARLES: Then it’s like, this is a very simple, very useful concept.

JULIE: Right.

CHARLES: And it’s two-sided. There on the one hand, the terminology is obtuse. But at the same time, it’s not. It’s just, it is what it is. And it’s just a symbol that’s referencing a concept.

JULIE: Right, right.

CHARLES: It’s a simple concept. So, I just want to be… I know for our listeners, I know that there’s a general admonition. Don’t worry about the terminology. It’s…

JULIE: Right, right. Like what I just said, I said the word ‘monad’. I just threw that out there at everybody, but [chuckles] it doesn’t matter which one of these words we’d be talking about or whatever I call them. We could give monads a different name and it’s still this concept that once you understand the concept itself, and then you can apply it in new situations, it doesn’t matter then what it’s called. But it does take getting used to. The words are… well, I think functor is a pretty good word for what it is. If you know the history of functor and how it came to mean what it means, I think it’s a pretty good word.

CHARLES: Really? So, I would love to know the history. Because functor is mystifying to me. It sounds like, I think the analogy I use is like if George Clinton and a funk parliament had an empire, the provinces, the governors of the provinces would be functors.

ELRICK: [Laughs]

JULIE: Yes.

CHARLES: But [Laughs] that’s the closest thing to an explanation I can come up with.

JULIE: I might use that. I’m about to give a talk on functors. I might use that.

[Laughter]

ELRICK: Isn’t that the name of the library? Funkadelic?

CHARLES: Well, that’s the name of the library that I’ve been…

JULIE: [Could be], yeah.

ELRICK: That you’d been…

CHARLES: That I’d been [writing] for JavaScript.

ELRICK: Yeah.

CHARLES: That imports all these concepts.

JULIE: [Laughs]

ELRICK: Yeah.

JULIE: Yeah.

ELRICK: So awesome.

JULIE: Yeah. Yeah, I have…

CHARLES: So, what is the etymology of functor?

JULIE: Well, as far as I can tell, Rudolf Carnap, the logician, invented the word. I don’t know if he got it from somewhere else. But the first time I can find a reference to it is in, he wrote a book about… he was a logician but this is sort of a linguistics book. It’s called ‘The Logical Syntax of Language’. And that’s the first reference I know of to the word functor. And he was trying to really make language very logically systematic, which natural language is and isn’t, right? [Chuckles]

CHARLES: Right.

JULIE: But he was only concerned with really logically systematizing everything. And so, he used the word functor to describe some kinds of function words in language that relate one part of a sentence to another part of a sentence.

CHARLES: Huh. So, what’s an example?

JULIE: So, the example that I’ve used in the past is, as far as I know this is not one that Carnap himself actually uses but it’s the clearest one outside of that book… well the ones inside the book I don’t really think are very good examples because they’re not really how people talk. So, the one that I’ve used to try to explain it is the word ‘not’ in English where ‘not’ gets applied to the whole sentence. It doesn’t really change the logical structure of the sentence. It doesn’t change the meaning of the sentence except for now it negates the whole thing.

CHARLES: I see.

JULIE: And so, it relates this sentence with this structure to a different context, which is now the whole thing has been negated.

CHARLES: I see. So, the meaning changes, but the structure really doesn’t.

JULIE: Right. And it changes the whole meaning.

CHARLES: Right.

JULIE: Not just part of the sentence. So, if you imagine ‘not’ applying to an entire sentence because of course we can apply it just to a single word or just to a single phrase and change the meaning just of that word or that phrase, but if you imagine a context where you’ve applied ‘not’ to a whole sentence, to an entire proposition, because of course he’s a logician. So, if you’ve applied ‘not’ to an entire proposition, then it doesn’t change the structure or the meaning of that proposition per se except for it just relates it to the category of negated propositions.

CHARLES: Mmhmm.

JULIE: So, that’s where it comes from. And…

CHARLES: But I still don’t understand why he called it functor.

JULIE: He’s sort of making up… well, actually I think the German might be the same word.

CHARLES: Ah, okay.

JULIE: Because he was writing in German. Because he’s looking for something that evokes the idea of ‘function word’.

CHARLES: Oh.

JULIE: So, if you were to take the ‘func’ of ‘function’ [Laughs] and the, I don’t know, maybe in German there’s some better explanation for making this into a particular word. But that’s how I think of it. So, it’s ‘function word’. And then category theorists took it from Carnap to mean a way to map a function in this category or when we’re talking about Haskell, a function of this type, to a function of another type.

CHARLES: Okay.

JULIE: And so, it takes the entire function, preserves the structure of the function just like negation preserves the structure of the sentence, and maps the whole thing to just a different context. So, if you had a function from A to B, functor can give you a function from maybe A to maybe B.

CHARLES: Right.

JULIE: So, it takes the function and just maps it into a different context.

CHARLES: Right. So, a JavaScript example is if I’ve got an array of ints and a function of ints to strings, I can take any array of ints and get an array of strings.

JULIE: Right.

CHARLES: Or if I have a promise that has an int in it, I can take that same function to get a promise of a string.

JULIE: Yeah.

CHARLES: Yeah. I had no idea that it actually came from linguistics.

JULIE: Yeah. [Laughs]

CHARLES: So actually, the category theorists even… it digs deeper than category theory. They were actually borrowing concepts.

JULIE: They were, yes.

CHARLES: We just always are borrowing concepts.

ELRICK: I like the borrowing of concepts.

JULIE: Yeah.

ELRICK: I think where people struggle with certain things, it’s tying it back to something that they’re familiar with. So, that’s where I get… my mind is like [makes exploding sound] “I now get it,” is when someone ties it back to something that I am…

CHARLES: Right.

ELRICK: Familiar with. Like Charles’ work with the JavaScript, tying it with JavaScript. I’m like, “Oh, now I see what they’re talking about.”

JULIE: Right.

CHARLES: because you realize, you’re using these concepts. People are using them, just they're using them anonymously.

JULIE: Right.

ELRICK: True.

CHARLES: They don’t have names for them.

JULIE: Right.

ELRICK: True.

CHARLES: It’s literally like an anonymous function and you’re just taking that lambda and assigning it to a symbol.

JULIE: Yeah.

CHARLES: You’re like “Oh wait. I’ve been using this anonymous function all over the place for years. I didn’t realize. Boom. This is actually a formal concept.”

ELRICK: True. And I think when people say like “Don’t reinvent the wheel” it’s a great statement for someone that has seen a wheel already.

[Laughter]

ELRICK: You know what I’m saying? If you never saw a wheel, then your’e going to reinvent the wheel because you’re like “Aw man. This doesn’t exist.” [Chuckles]

JULIE: Yeah.

ELRICK: But if people are exposed to these concepts, then they wouldn’t reinvent the wheel.

CHARLES: Right.

JULIE: Right. Yeah.

CHARLES: Instead of calling in some context, calling it a roller. [Chuckles] It’s a round thingy.

[Laughter]

JULIE: Right. Yeah, so that’s a little bit what I tried to do in my monoid talk in London. I tried to give some history of monoid, where this idea comes from and why it’s worth talking about these things.

CHARLES: Yeah.

JULIE: Why it’s worth talking about the structure.

CHARLES: So, why is it worth the… where did it come from and why is it worth talking about?

JULIE: Oh, so back when Boole, George Boole, when he decided to start formalizing logic…

CHARLES: George Boole also, he was a career-switcher too, right? He was a primary school teacher.

JULIE: Right, yeah.

CHARLES: If I recall. He actually, he was basically teaching. Primary school is like elementary school in England, right?

JULIE: I believe so, yes.

CHARLES: Yeah. I think he was like, he was basically the US equivalent of an elementary school teacher who then went on to a second and probably, thankfully a big career that left a big legacy.

JULIE: Right. Although no one knew exactly how big the legacy was really, until Claude Shannon picked it up and then just changed the whole world.[Laughs] Anyway, so Boole, when he was trying to come up with a formal algebra of logic so that we could not care so much about the semantic content of arguments (we could just symbolize them and just by manipulating symbols we could determine if an argument was logically valid or not), he was… well, for disjunction and conjunction which is AND and OR – well, disjunction would be the OR and conjunction the AND – he had prior art. He had addition and multiplication to look at. So, addition is like disjunction in some important ways. And multiplication is like conjunction in some important ways.

CHARLES: Mmhmm. No, it doesn’t.

JULIE: [Inaudible] like a logical OR. So, it took me a while to see that. But they’re also related then to set intersection and union where intersect-…

CHARLES: So can… Let’s just stop on that for a little bit, because let me parse that. So, for OR I’ve got two values, like in an ‘if’ statement. This OR that. If I’ve got a true value then I can OR that with anything and I’ll get the same anything.

JULIE: Right.

CHARLES: So, true is the identity value of OR, right? Is that what you’re saying? So, one…

JULIE: Well, it’s false that’s the identity of OR.

CHARLES: Oh, it is?

JULIE: Zero is the identity of addition.

CHARLES: Wait, but if I take ‘false OR one’ I get… oh, I get one.

JULIE: Right.

CHARLES: Okay. So, if I get ‘false OR true’, I get true. Okay, so false is the identity.

JULIE: Yeah.

CHARLES: Oh right. You’re right. You’re right. Because… okay, sorry.

JULIE: So, just like in addition, zero is the identity. So, whatever you add to zero, that’s the result, right? You’re going to get [the same]

CHARLES: Right.

JULIE: Value back. So, with OR false is the identity and false is equivalent to zero.

CHARLES: [Inaudible] ‘False OR anything’ and you’re getting the anything.

JULIE: Right. So, the only time you’ll get a false back is if it’s ‘false OR false’, right?

CHARLES: Right. Mmhmm.

JULIE: Yeah. So, false is the identity there. And then it’s sort of the same for conjunction where one is the identity of multiplication and one is also the… I mean, true is then the identity of logical conjunction.

CHARLES: Right. Because one AND…

JULIE: ‘True AND false’ will get the false back. [Inaudible]

CHARLES: Right. ‘True And true’ you can get the true back.

JULIE: Yeah.

CHARLES: Okay.

JULIE: And it’s also then true, getting back to what we were talking about, semirings, it’s also true that false is a kind of annihilator for conjunction. That’s sort of trivial, because…

CHARLES: Oh, because you annihilate the value.

JULIE: Right. When there’s only two values it’s a little bit trivial. But it is [inaudible]. So…

CHARLES: But it’s [inaudible]. Yeah. It demonstrates the point.

JULIE: Right.

CHARLES: So, if I have yeah, ‘false AND anything’ is just going to be false. So, I annihilate whatever is in that position.

JULIE: Right.

CHARLES: And the same thing as zero is the annihilator for multiplication, right?

JULIE: Right.

CHARLES: Because zero times anything and you annihilate the value.

JULIE: Yeah.

CHARLES: And now I’ve got… okay, I’m seeing it. I don’t know where you’re going with this.

[Laughter]

ELRICK: Yeah.

CHARLES: But I’m there with you.

ELRICK: Yup.

JULIE: And then it turns out there are some operations from set theory that work really similarly. So, intersection and union are similar but the ones that are closer to conjunction/disjunction are disjoint unions and cartesian products. So we don’t need to talk about those a whole lot if you’re not into set theory. But anyway…

CHARLES: I like set theory although it’s so hard to describe without pictures, without Venn diagrams.

JULIE: It is. It really is, yeah. So anyway, all of these things are monoids. And they’re all binary associative operations with identity elements. So, they’re all monoids. And so, we’ve taken operations on sets, operations on logical propositions, operations on many kinds of numbers (because not all kinds of addition and multiplication I guess are associative), and we can kind of unify all of those into the same framework. And then once we have done that, then we can see that there’s all these other ‘sets’. Because most of the kinds of numbers are sets and there are operations on generic sets with set theory. So, now we can say “Oh. We can do these same kinds of operations on many other kinds of sets, many other varieties of sets.” And we can see that same pattern. And then we can get a kind of intuition for “Well, if I have a disjunctive monoid where I’m adding two things or I’m OR-ing two things…” Because even though those are logically very similar, intuitively and in terms of what it means to concatenate lists versus choosing one or the other, those obviously have different practical effects.

CHARLES: So, I’m going to try and come up with some concrete examples to maybe…

JULIE: Okay, yeah.

CHARLES: A part of them will probably be like in JavaScript, right? So, to capture the idea of a disjunctive monoid versus a conjunctive monoid. So, a disjunctive monoid is like, so in JavaScript we’re got two objects. You concat them together and it’s like two maps or two hashes. So, you mash them together and you get… so, for the disjunctive one you’d have all the keys from both of the hashes inside the resulting object. You take two objects. Basically we call it object assign in JavaScript where you have basically the empty object. You can take the empty object and then take any number of objects. And so, we talked about…

JULIE: That would become a disjunctive monoid, right?

CHARLES: That would be a disjunctive monoid because you’re like basically, you’re OR-ing. Yeah.

JULIE: You’re kind of, [inaudible]

CHARLES: Hard to find the terminology.

JULIE: Yeah.

CHARLES: But like object assign would be a disjunctive monoid because you’re like mashing these two objects. And the resulting object has all of the things from both of them.

JULIE: Right. So, it’s like a sum of the two, right?

CHARLES: Right, right. Okay, so then another one would be like min or max where you’ve got this list of integers and you can basically take any two integers and you can mash them together and if you’re using min, you get the one that’s smaller. Basically, you’re collapsing them into one value but you’re actually just choosing one of them. Is that like…

JULIE: Yeah.

CHARLES: Would that be like a conjunctive monoid?

JULIE: No, that’s also disjunctive but that’s more like an OR than like a sum.

CHARLES: Okay.

JULIE: Right. So, that’s what I said. It’s hard to think of disjunctive monoids I think because there’s really two varieties. There’s some underlying logical similarity, like the similarity in the identity values. But they’re also different. Summing two things versus choosing one or the other are also very different things in a lot of ways.

CHARLES: Right. Okay.

JULIE: And so, I think the conjunctive monoids are all a little bit more similar, I think. [Chuckles] But the disjunctive monoids are two broad categories. And we don’t really have a monoid in Haskell of lists where you’re choosing one or the other. The basic list monoid is you’re concatenating them. So, you’re adding two lists or taking the union of them. But for maybe, the maybe type, we do have monoids in Haskell where you’re just choosing either the first just value that comes up or the last just value that comes up. So, we do have a monoid of choice over the maybe type. And then we have a type class called alternative which is monoids of choice for… so, they’re disjunctive monoids but instead of adding the two things together, they’re choosing one or the other.

CHARLES: Okay.

JULIE: Though we have a type class for that. [Laughs]

CHARLES: [Sighs] Oh wow. Yeah.

JULIE: Mmhmm, yeah.

CHARLES: I’l have to go read up on that one.

JULIE: That type class comes up the most when you’re parsing, because you can then parse… like if you found this thing, then parse this thing. But if you haven’t found this thing, then you can keep going. And if you find this other thing later, then you can take that thing. So, you allow the possibility of choice. The first thing that you come to that matches, take that thing or parse that thing. So, that type class gets mostly used for parsing but it’s not only useful for parsing.

CHARLES: Okay.

JULIE: So yeah. That’s the most of the time when I’ve used it.

CHARLES: Is this when you’re like parsing JSON? Or is this when you’re just searching some stream for some value? Like you just want to run through it until you encounter this value? Or how does that…?

JULIE: Right. Say you want to run through it until you find either this value or this value. I’ve used it when I’ve been parsing command line arguments. So, let’s say I have some flags that can be passed in on my command line command. There are some flags that could be passed in. So, we’ll parse until we find this thing or this thing. This flag or this flag. So, if you find this flag, then we’re going to go ahead and parse that and do whatever that flag says to do. If you don’t find that first flag then we can keep parsing and see if you find this other flag, in which case we’ll do something different.

CHARLES: Okay.

JULIE: It’ll take the first match that it finds. Does that make sense?

CHARLES: Yeah, yeah, yeah. It does. But I’m not connecting how it’s a monoid. [Laughs]

JULIE: How is that a monoid? Well, because it’s a monoid of OR-ing

CHARLES: What’s the identity value or the empty value in that case?

JULIE: Well, the empty value would be… let’s say you have maybes. Let’s say you have some kind of maybe thing, so you’re parser is going to return maybe this thing, maybe whatever you’re parsing. Like maybe string.

CHARLES: Yeah, yeah.

JULIE: So, it’s going to return a maybe string. So well, nothing would be the empty.

CHARLES: Okay.

JULIE: But nothing is like the zero because it’s a disjunction, logical OR. So, only when you have two nothings will you get back a nothing. Otherwise, it will take the first thing that it finds.

CHARLES: Okay. I see.

JULIE: Yeah. So, the identity then is the nothing, like false is the identity for disjunction.

CHARLES: Mmhmm. Okay.

JULIE: Yeah.

CHARLES: [Inaudible]

JULIE: Yeah. If you have nothing or this other thing, then you return this other thing. Then you return the maybe string. If you have two nothings, then you get in fact nothing. Your parsing has failed.

CHARLES: Right, because you’ve got nothing.

JULIE: Because you’ve got nothing. There was nothing to give you back.

CHARLES: So, you concatenated all of the things together and you ended up with nothing.

JULIE: Right, because there was nothing there.

CHARLES: Right. [Laughs]

JULIE: You found nothing. So, it’s useful when you’ve got some possibilities that could be present and you just want to keep parsing until you find the first one that matches. And then it’ll just return whatever. It’ll just parse the first thing that it matches on.

CHARLES: Okay, okay.

JULIE: Does that make sense?

CHARLES: Yeah. No, I think it makes sense.

JULIE: I’m not sure. Because I feel like I kind of went down a rabbit hole there. [Laughs]

CHARLES: Yeah. [Laughs] No, no. I think it makes sense. And as a quick aside, I think… so, I was, when we were talking about min and max, are min and max also like a semiring? Because negative infinity is the annihilator of min and it’s the identity of max. and positive infinity is the annihilator of max but it’s the identity of min.

JULIE: I guess. I don’t really think of min and max as having identities. Is that how [inaudible]?

CHARLES: I’m just, I don’t know. Well, I think if you have negative infinity and you max it with anything, you’re going to get the anything, right? Negative infinity max one is one. Negative infinity/minus a billion is minus a billion.

JULIE: Yeah, okay.

CHARLES: I don’t know. Just off the cuff. I’m just trying to… annihilators sound cool. And so…

[Laughter]

CHARLES: And so I’m like, I’m trying to find annihilators.

JULIE: Yeah, they are cool.

CHARLES: [Laughs]

JULIE: One of my friends on Twitter was just talking about how he used the intuition at least of a semiring at work because he had this sort of monoid to concatenate schedules. So, he’s got all these different schedules and he’s got this kind of monoid to concatenate them, to merge the schedules together. But then he’s got this one schedule that is special. And whenever something is in this schedule, it needs to hard override every other schedule.

CHARLES: Right.

JULIE: And so, that was like the annihilator. So, he was thinking of it as a semiring, because that hard override schedule is like the annihilator of all the other schedules.

CHARLES: Yeah.

JULIE: If anything else exists on this day or whatever, then it’d just get a hard override. So, there’s a real world use. [Laughs]

CHARLES: Yeah, a real world example. That’s the thing that I’m finding, is that all these really very crystalline abstractions, they still play out very well I think in the real world. And they’re useful as a took in terms of casting a net over a problem. Because you’re like… when I’m faced with something new, I’m like “Well, let’s see. Can I make it a functor?” And if I can, then I’ve unlocked all these goodies. I’ve unlocked every single composition pattern that works with functor.

JULIE: Right, right.

CHARLES: And it’s like sometimes it fits. It almost feels like when you’re working on something at home and you’ve got some bolt and you’re trying on different diameters. So you’re like, “Oh, is it 15 millimeter? Is it 8 millimeter?”

JULIE: Right. [Laughs]

CHARLES: “Like no, okay. Maybe it’ll work with this.” But then when it clicks, then you can really ratchet with some serious torque.

JULIE: Right, right. Yeah.

CHARLES: So, yeah. Definitely trying to look for semirings [Laughs] is definitely beyond my [can] at this point. But I hope to get there where it can be like, if it’s a fit, it’s a fit. That’s awesome.

JULIE: Right. Yeah, it’s kind of beyond my can too. Semirings are still a little bit new for me and I can’t say that I find them in the wild as it were, as often as monoids or something. But I think it just takes seeing some concrete examples. So, now you know this idea exists. If you just have some concrete examples of it, then over time you develop that intuition, right?

CHARLES: Right.

JULIE: Like “Okay, I’ve seen this pattern before.” [Chuckles]

CHARLES: yeah. Basically, every time now I want to fold a list, or like in JavaScript, any time you want to reduce something I’m like “There’s a monoid here that I’m not seeing. Let me look for it.”

JULIE: Yeah. Oh, that’s cool, yeah.

CHARLES: Because like, that’s basically, most of the time you’re doing a reduce, then like I said that’s the terminology for fold in JavaScript, is you start with some reducible thing. Then you have an initial value and a function to actually concatenate two things together.

JULIE: Right.

CHARLES: And so, usually that initial state, that’s your identity. And then that function is just your concat function from your monoid. And so, usually anytime I do a reduce, there’s the three pieces. Boom. Identity value, concatenation function, it’s usually right there. And so, that’s the way I’ve found of extracting these things, is I’m very suspicious every time I’m tempted to…

JULIE: [Laughs]

CHARLES: A fold. I’m like “Hmm. Where’s the monoid I’m missing? Is it [under the] couch?” Like, where is it? [Laughs] Because it just, it cleans it up and it makes it so much more concise.

JULIE: Oh yeah, that’s awesome.

CHARLES: So anyhow.

JULIE: Have we totally lost Elrick?

ELRICK: Nope, I’m still here.

JULIE: Okay. [Laughs]

ELRICK: I’m sitting in and listening to you two break down these complex topics is really good. Because you guys break them down to a level where it’s consumable by people that barely understand it. So, I’m just sitting here just soaking everything in like “Oh, that’s awesome.” Taking notes. Yes, okay, okay.

[Laughter]

JULIE: Cool.

ELRICK: So, I’m like riding the train in the back just hanging out, feeling the cool breeze while you guys just pull the train ahead in…

[Laughter]

ELRICK: In the engine department, you know? It’s awesome.

CHARLES: Yeah.

ELRICK: I don’t know if they’re related. But you were talking about semirings and I heard of semigroups or semigroups. I have no idea if those two things are related. Are they related or [inaudible]?

JULIE: They're kind of related. So, a semigroup is like a monoid but doesn’t have an identity value.

CHARLES: What is an example of a semigroup out there in the wild? Because every time I find a semigroup, I feel like it’s actually a monoid.

JULIE: Well, you know I feel like that a lot, too. We do have a data type in Haskell that is a non-empty list. So, there is no empty list

CHARLES: Ah, right. Okay.

JULIE: So then you can concatenate those lists, but there’s never an identity value for it.

CHARLES: I see.

JULIE: Yeah. So, that’s a case. There’s actually a lot of comparison functions, greater than and less than. I think those are semigroups because they’re binary, they’re associative, but they don’t have an identity value. Like if you’re comparing two numbers, there’s not really an identity value there.

CHARLES: Right. Well, would the negative infinity work there? Let’s see. Like, negative infinity greater than anything would be the anything. Well, okay wait. But greater than, that takes numbers and yields a boolean, right?

JULIE: Yeah,

CHARLES: Right. So, it couldn’t be… could it be a semigroup? Don’t semigroups have to… Doesn’t the [inaudible] function have to yield the same type as the operands?

JULIE: Yes.

CHARLES: But a non-empty list, that’s a good one. Sometimes it’s basically not valid for you to have a list that doesn’t have any elements, right? Because it’s like the null value or the empty value and it could be like a shopping cart on Amazon. You can’t have a shopping cart without at least something in it.

JULIE: Right.

CHARLES: Or, you can’t check out without something. So, you might want to say like the shopping cart that I’m going to check out is a non-empty list. And so, you can put two non-empty lists together. But yeah, there’s no value you can mash together, you can concat with anything, that isn’t empty.

JULIE: Right.

CHARLES: So, I guess going back to your question Elrick, I don’t know if it’s related to semiring. But semigroup is just, it’s like one-half of monoid. It’s the part that concats two values together.

JULIE: Right. Well, yeah. And so, it’s supposed to be half a group, right? But I don’t remember…

CHARLES: [Laughs]

JULIE: [Inaudible] all of the group stuff is, all the stuff that these types have to have to be a group. And similarly, I forget what the difference between semiring and ring is. [Chuckles] Because a ring and a group I know are not the same thing. But I forget what the difference is, too. So, I kind of got a handle on what semigroups are, and I know all my Haskell friends are going to, when they hear this podcast they’re going to tweet all these examples of semigroups at me, especially my coauthor for ‘Joy of Haskell’, Chris Martin. He’s really into semigroups. And so, I know he’s going to be very disappointed in my inability to think…

[Laughter]

JULIE: To think of any good examples. But it’s not something that I find myself using a lot, whereas semirings are something that I have started noticing a little bit more often. So, how a monoid relates to a group is something that I can’t remember off the top of my head. And I know how semirings relate to monoids, but how monoids then relate to rings and groups, I can’t really remember. And so, these things are sort of all related. But the relation is not something I can spill out off the top of my head. Sorry. [Laughs]

CHARLES: No, It’s no worries. You know, I feel like…

ELRICK: It’s all good.

CHARLES: What’s funny is I feel like having these discussions is exactly like the discussions people have with any framework of using one that we use a lot, which is EmberJS. But if you could do with React or something, it’s like, how does the model relate to the controller, relate to the router, relate to the middleware, relate to the services? You just have these things, these moving parts that fit together. And part of… I feel like exploring this space is really, absolutely no different than exploring any other software framework where you just have these things, these cooperating concepts, and they do click together. But you just have to map out the space in your head.

JULIE: Yeah. This is going to sound stupid because everybody thinks that because I know Haskell I must know all these other things. But I just had to ask people to recommend me a book that could explain the relationship of HTML and CSS, because that was completely opaque to me.

CHARLES: [Laughs] Yeah.

JULIE: I’ve been involved in the making now of several websites because of the books and stuff like that. And I have a blog. It’s not WordPress or anything. I did that sort of myself. So, I’ve done a little bit with that. But CSS is really terrifying. And…

CHARLES: Right. Like query selectors, rules, properties.

JULIE: Yeah.

ELRICK: [Laughs]

CHARLES: Again, might as well be groups and semigroups and monoids, right?

JULIE: Right, right.

ELRICK: Yeah.

CHARLES: [Laughs]

ELRICK: That is really interesting. [Chuckles] I’ve never heard anyone make that comparison before. But it’s totally true, now that I’m thinking about it.

JULIE: Yeah, yeah.

CHARLES: Yeah. In the tech world we are so steeped in our own jargon that we could be… we can reject one set of jargon and be totally fine with another set. Or be like, suspicious of one set of concepts working together and be totally fine with these other designations which are somewhat arbitrary but they work.

JULIE: Right.

CHARLES: So, people use them.

JULIE: So, it’s like what you’ve gotten used to and what you’re familiar with and that seems normal and natural to you. [Chuckles] So, the Haskell stuff, most of it seems normal and natural to me. And then I don’t understand HTML and CSS. So, I bought a book.

[Laughter]

CHARLES: Learning HTML and CSS from first principles.

JULIE: Yes, yeah. I just wanted to understand. I could tell that they do relate to each other, that there is some way that they click together. I can tell that by banging my head against them repeatedly. But I didn’t really understand how, and so yeah. So, i’ve been reading this book to [Laughs] [learn] HTML and CSS and how they relate together. That’s so important, just figuring out how things relate to each other, you know?

CHARLES: Yeah.

ELRICK: Yeah. That is very true.

JULIE: Yeah.

ELRICK: We can trade. I can teach you HTML and CSS and you can teach me Haskell.

JULIE: Absolutely.

ELRICK: [Laughs]

CHARLES: There you go

JULIE: [Laughs]

ELRICK: Because I’m like, “Ooh.” I’m like, “Oh, CSS. Great. No problem.”

[Laughter]

ELRICK: Haskell, I’m like “Oh, I don’t know.”

JULIE: Yeah.

CHARLES: Yeah.

ELRICK: [Laughs]

CHARLES: No, it’s amazing [inaudible] CSS.

ELRICK: Yeah.

CHARLES: It is, it’s a complicated system. And it’s actually, it’s in many ways, it’s actually a pretty… it’s a pretty functional system, CSS is at least. The DOM APIs are very much imperative and about mutable state. But CSS is basically yeah, completely declarative.

JULIE: Right.

CHARLES: Completely immutable. And yeah, the workings of the interpreter are a mystery. [Laughs]

ELRICK: Yup.

JULIE: YEs. And you know, for the Joy of Haskell website we use Bootstrap. And so, there was just like… there’s all this magic, you know? [Laughs]

ELRICK: Oh, yeah.

CHARLES: Yeah.

JULIE: Oh look, if I just change this little thing, suddenly it’s perfectly responsive and mobile. Cool.

[Laughter]

JULIE: I don’t know how it’s doing this, but this is great. [Laughs]

CHARLES: Yeah. Oh, yeah. It’s an infinite space. And yeah, people forget what is so easy and intuitive is not and that there’s actually a lot of learning that happened there that they’re just taking for granted.

JULIE: I think so many people start from HTML and CSS. That’s one of their first introductions to programming, or JavaScript or some combination of all three of those. And so, to them the idea that you would be learning Haskell first and then coming around and being like “Oaky, I have to figure out HTML,” that [seems very] strange, right?

[Laughter]

CHARLES: Yeah. Well, definitely probably stepping into bizarro world.

JULIE: And I went backwards. But [Laughs]

CHARLES: Yeah.

JULIE: Not that it’s backwards in terms of… just backwards in terms of the normal way, progression of [inaudible]

CHARLES: Yeah. It’s definitely the back door. Like coming in through the catering kitchen or something.

JULIE: Yes.

CHARLES: Instead of the front door. Because you know the browser, you can just open up the Dev Tools and there you are.

JULIE: Exactly, yeah.

CHARLES: The level of accessibility is pretty astounding. And so, I think t’s why it’s one of the most popular avenues.

JULIE: Oh, definitely. Yeah.

ELRICK: It’s the back door probably for web development but not the back door for programming in general.

JULIE: Mm, yeah. Yeah.

CHARLES: Yeah. It seems like Haskell programming has really started taking off and that the ecosystem is starting to get some of the trappings of a really less fricative developer experience in terms of the package management and a command line experience and being able to not make all of the tiny little decisions that need to be made before you’re actually writing ‘hello world’.

JULIE: Right.

ELRICK: Interesting. Haskell has a package manager now?

CHARLES: Oh, it has for a while.

ELRICK: Oh, really? What is it called? I have no idea? Do you know the name off the top of your head?

CHARLES: So, I actually, I’m not that familiar with the ecosystem other than every time I try it out. So I definitely will defer this question to you, Julie.

JULIE: This is going to be a dumb question, I guess. What do we mean by package manager?

CHARLES: So, in JavaScript, we have npm. The concept of these packages. It’s code that you can download, a module that you can import, basically import symbols from. And Ruby has RubyGems. And Python has pip.

JULIE: Okay, okay.

CHARLES: Emacs has Emacs Packages. And usually, there’s some repository and people could publish to them and you can specify dependencies.

JULIE: Right, yeah. Okay, so we have a few things. Hackage is sort of the main package repository. And then we have another one called Stackage and the packages that are in Stackage are all guaranteed to work with each other.

CHARLES: Mm, okay.

JULIE: So, on Hackage, some of the packages that are on Hackage are not really maintained or they only work with some old versions of dependencies and stuff like that, so the people who made Stackage were like “well, if we had this set of packages that were all guaranteed to work together, the dependencies were all kept updated and they all can be made to work together, then that would be really convenient.” And then we have Cabal and we have Stack are the main… and a lot of people use Nix for the same purpose that you would use Cabal or Stack for building projects and importing dependencies and all of that.

CHARLES: Right. So, Cabal and Stack would be roughly equivalent then to the way we use Yarn or JavaScript and Bundler in Ruby. You’re solving the equation for, here’s my root set of dependencies. Go out and solve for the set of packages that satisfy. Give me at least one solution and then download those packages and [you can] run them.

JULIE: Yeah, yeah. Right, so managing your dependencies and building your project. Because Haskell’s compiled, so you’ve got to build things. And so yeah, we have both of those.

CHARLES: And now there’s like web frameworks and REST frameworks.

JULIE: Oh there are, yeah. We have…

CHARLES: All kinds of stuff now.

JULIE: We had this big proliferation of web frameworks lately. And I guess some of them are very good. I don't really do web development. But the people I know who do web development in Haskell say that some of these are very good. Yesod is supposed to be very good. Servant is sort of the new hotness. And I haven’t used Servant at all though, so don't ask me questions about it.

[Laughter]

JULIE: But yeah, we have several big web frameworks now. There are still some probably big holes in the Haskell ecosystem in terms of what people want to see. So, that’s one thing that people complain about Haskell for, is that we don’t have some of the libraries they’d like to see. I’d like to see something… I would really like to see in Haskell something along the lines of like NLTK from Python.

CHARLES: What is that?

JULIE: Natural language toolkit.

CHARLES: Oh, okay.

JULIE: So yeah, Python has this…

CHARLES: Yeah, Python’s got all the nice science things.

JULIE: They really do. And Haskell has some natural language processing libraries available but nothing along the lines of, nothing as big or easy to use and stuff as NLTK yet. So, I’d really like to see that hole get filled a little bit better. And you know…

CHARLES: Well, there you go. If anyone out there is seeking fame and fortune in the Haskell community.

JULIE: That’s actually why I started learning Python, was just so that I could figure out NLTK well enough to start writing it in Haskell.

[Laughter]

JULIE: So, that’s sort of my ambitious long-term project. We’ll see how that goes. [Laughs]

CHARLES: Nice. Before we wrap up, is there anything going on, coming up, that you want to give a shoutout to or mention or just anything exciting in general?

JULIE: Yeah, so on March 30th I’m going to be giving a talk at lambda-squared which is going to be in Knoxville and is a new conference. I think it’s just a single-day conference and I’m going to be giving a talk about functors. So, I’m going to try to get through all the exciting varieties of functors in a 50-minute talk.

CHARLES: Ooh.

JULIE: So, we’ll see how that goes. Yeah. And I am still working with Chris Martin on ‘The Joy of Haskell’ which should be finished this year, sometime. I’m not going to…

[Laughter]

JULIE: Give any more specific deadline than that. And in the process of writing Joy of Haskell, I was telling him about some things that, some things that I think are really difficult. Like in my experience, teaching Haskell some places where I find people have the biggest stumbling blocks. And I said, “What if we could do a beginner video course where instead of throwing all of these things at people at once, we separated them out?” And so, you can just worry about this set of stumbling blocks at one time and then later we can talk about this set of stumbling blocks. And so, we’re doing… we’re going to start a video course, a beginner Haskell video course. I think we’ll be starting later this month. So, I’m pretty excited…

CHARLES: Nice.

JULIE: About that. Yeah.

CHARLES: Yeah, I know a lot of people learn really, really well from videos. There’s just some…

JULIE: Yeah. [Inaudible] for me, so I’m a little nervous. But [Laughs]

CHARLES: Yeah, especially if you can do… are you going to be doing live coding examples? Building out things with folks?

JULIE: Yeah.

CHARLES: Yeah. Well, you just needn’t look no further than the popular things like RailsCasts and some of the… yeah, there’s just so many good video content out there. Yeah, we’ll definitely be looking for the.

JULIE: Cool.

CHARLIE: Alright. Well, thank you so much, Julie, for coming on.

JULIE: Well, thank you for having me on. Sorry I went down some… I went kind of down some rabbit holes. Sorry about that. [Laughs]

CHARLES: You know what? You go down the rabbit holes, we spend time walking around the rabbit holes.

JULIE: [Laughs]

CHARLES: There’s something for everybody. So…

[Laughter]

CHARLES: And ultimately we’re strolling through the meadow. So, it’s all good.

JULIE: [Laughs] Yeah.

CHARLES: Thank you too, Elrick.

JULIE: It was nice talking to you guys again.

CHARLES: Yeah.

ELRICK: Yeah, thank you.

CHARLES: If folks want to follow up with you or reach out to you, what’s the best way to get in contact with you?

JULIE: I’m @argumatronic on Twitter and my blog is argumatronic.com which has an email address and some other contact information for me. So, I’d love to hear questions, comments. [Laughs] Yeah. I always [inaudible].

CHARLES: Alright, fantastic.

JULIE: To talk to new people.

CHARLES: Alright. And if you want to get in touch with us, we are @TheFrontside on Twitter. Or you can just drop us an email at [email protected]. Thanks everybody for listening. And we will see you all later.

Sam Cates: @SamCates | GE Ventures

Show Notes:

Resources:

Transcript:

CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 92. My name is Charles Lowell, a developer here at The Frontside and I am your podcast host-in-training kicking it off in 2018. [Inaudible] of our first episode. We’ve got Elrick also joining us. Hello, Elrick.

ERICK: Hey. How you doing, Charles?

CHARLES: I’m doing well. I’m doing well. You having a good new year so far?

ERICK: Yeah, it’s great. There’s a snowstorm passing through today. So, I’m going to break in the New Year shoveling.

CHARLES: Let us know if we need to parachute in some shovels for you.

ERICK: [Laughs]

CHARLES: And then with us today, we have Sam Cates on the show who is… a lot of times we have developers on the show. He’s actually a venture… what would you describe yourself as?

SAM: Yeah, I’d say I’m a venture investor with GE Ventures. So, on the corporate investing side.

CHARLES: Okay. Now, I didn’t even know that GE actually had a corporate investing side. Is that pretty common for a large company?

SAM: You know, it’s becoming increasingly common. I think in 2015 there was actually a peak of activity coming from corporate venture capital groups. And I’ve only seen the number of firms escalate since then. Although the dollars invested stays pretty consistent. But if you look at a lot of big companies, particularly in the common tech world like Cisco, Google, Intel, they have historically had large venture firms inside of themselves. And then GE and a lot of other industrials have since followed suit. We’ve been at it for about five years and we see it increasingly.

CHARLES: And so, have you been with them since the beginning?

SAM: Yeah, just about. I’ve actually been with GE for about nine years now. So, I was on the operating side in a number of the industrial businesses before I joined GE Digital and then GE Ventures. And so, it was just after GE Ventures got kicked off.

CHARLES: Oh, that’s exciting. So, what is it… now, we actually got connected to you through one of the companies that you actually invested in. It’s something that we use and we’re very interested in. Why don’t you tell us a little bit about what your job looks like on a day-to-day basis and what companies you invest in?

SAM: Sure. I really focus a lot of my time on Internet of Things companies. So, that’s a really big trend that GE has been a part of and a leader in over the past few years. And so, we spend time investing in companies that are directly working with GE or playing in similar spaces to us. And so, Elrick and I actually met at a hackathon for one of those companies. And I always like to use that as an example because it’s a good one, to demonstrate the kinds of investments we make. And that’s Resin.io. I know you guys have done an episode or two talking with them. But that for example was a ‘Series A’ investment that we made about two years ago. And then company essentially helps developers build connected products. And so, that’s something that GE cares a lot about. We had people inside the company who found the product and loved it and that’s actually how we met.

CHARLES: When you say ‘Series A’, can you give a brief overview of what the different stages of funding of a startup might be?

SAM: Yeah, yeah, certainly. So, maybe if I take a step back and answer your original question on what I do on a day-to-day basis. A lot of my job is meeting with all kinds of new companies, whether they be early stage, usually things that would be seed funding – and we’ll go into what some of those things mean – all the way through the late stage which would be companies that are maybe on the border of going public or are already profitable. And so, if we go into what kinds of investors there are, I think that’s probably an interesting subject to talk more about. But they’re a whole wide variety. When I said ‘Series A’ I just meant a company that was at what we would call the ‘Series A’ stage, and the letters act just like you’d expect. So, there’s ‘Series A’, ‘Series B’, ‘Series C’, and so on. And they all, they tend to look similar at those stages in terms of sizes and progress. But there is a range, and no two company is the same.

ERICK: In today’s world, it’s very easy for people to create a startup. They can write some code and they can either come up, get a Raspberry Pi or some microcontrollers or whatever it is, and either do an IoT startup or a software startup. Now, when you get to the point where you have an idea and you kick it off initially, how do you go about then saying, “Let me get some funding.” How do you even get funding?

SAM: Sure, yeah. And to your point, there’s a huge range of technologies that are making it easier to start almost any kind of company. It’s a great time to be an entrepreneur, whether it be 3D printing for hardware products, all the technologies that you were mentioning, AWS, all this stuff is contributing to reducing the cost to allow companies or people to create companies. And so, once people have gone out and experimented with some of these things and built what they think is a product the market wants, often if they require more money which may be for acquiring customers through things like Facebook Ads or simply doing further product development to make sure the product is somewhere that more customers could use it, often they can’t finance it just through their own revenue. And so, there are typical stages and types of investors that people go approach looking for money.

ERICK: Okay. What are those Series? I remember you mentioned something like a ‘Series A’ investment. So, initially when you’re looking for an investment, is that where you would… category you would be in as a startup looking for investment? They would consider you a ‘Series A’ startup?

SAM: Well, I want to caveat and just say every company is different. So, I see companies that…

ERICK: Gotcha.

SAM: Start out at a much later stage because they’re able to bootstrap to that point. And bootstrap is the word that I use for a company that funds its own investment. They get paid by customers and they use that money to continue building the product. But if I talk about the range of types of financing a company may go for, I think the way that most people categorize this, first people often raise from friends and family or angels. And so, it’s just money to get off the ground and maybe to pay the rent while you’re doing some of that experimenting we were talking about.

ERICK: Gotcha.

SAM: There can be some later rounds when they’d be ‘Series F’ or even beyond, I guess.

CHARLES: Right. So now, what are generally the terms on these? So, for my angel investments or my seed investments, I assume what distinguishes these is essentially how much ownership of the company you’re getting for how much money. And those kind of, those change as the product solidifies.

SAM: Yeah.

CHARLES: And the potential becomes more visible.

SAM: Yeah, it’s a wide… and again, these are all… the venture is a world of ranges. There’s a really wide difference between the two ends of any spectrum. So, I’ll just talk in generalities though. So, I think the latest report that I’ve seen at least for an annual basis was PitchBook’s 2016 report. And they were laying out some of the medians. So, for seed stage deals I believe it was something like one and a half million dollars raised was the median on a pre-money valuation of six and a half million. And that just means the company is worth, investors say the company is worth six and a half million dollars today. And we’re going to give you a million and a half dollars invested at that price.

CHARLES: So roughly, a sixth… they would take a sixth of the company then in return?

SAM: Yeah.

CHARLES: Okay.

ERICK: Ah.

CHARLES: Okay.

ERICK: Okay.

CHARLES: I see. That makes sense. So now, back to Elrick’s original question. If I’m, I’ve got my product. Or I’ve got this idea. I’ve written some code. I’ve turned it into a prototype product. Maybe I’m moving through these various stages. What type of VC am I going to be looking for? How do I actually find the right type to be talking to? I guess what types are there even?

SAM: Yeah. And one part… we mentioned a lot of the technologies that are making it easier to start companies. One part that also makes it easier is the proliferation of financing options, whether it be even more investors in these traditional structures we talked about like seed and A. And then there are other options that are emerging, things like you see a lot of people raising through what they call Initial Coin Offerings or ICOs. And then there are also things like AngelList which are attempting to democratize the investing process, make it more accessible.

CHARLES: So, there were two acronyms in there, or two specific technologies.

[Chuckles]

CHARLES: You talked about ICOs which I assumed that you said it was Initial Coin Offering. Not like insane clown offering.

[Laughter]

CHARLES: Which I would love to see. And then AngelList. So traditionally, these had been very network-based which brings to mind the capitalists of Old England or whatever where there’s a bunch of people with cigars in a room and I realize it’s not actually like that. What are each of these things? The AngelList and the ICOs? And how do they democratize that process?

SAM: It’s funny you should mention the old times. I think a good example of that is there are a lot of stories about the founding of General Electric. It’s a 126-year-old company and back then it was largely, it was Thomas Edison working with I believe was JP Morgan to get it off the ground. And so, today there’s still a bit of the network piece you’re mentioning. But I think of AngelList as a place that you can essentially market to investors. If you think about the types of people that are on there, it’s people that are looking to invest money in early stages in startups. And I’m not a big user of AngelList because I tend to be investing a little bit later. So, I really recommend anybody who’s interested, just go check it out. It’s I believe just Angel.co.

CHARLES: And what about an ICO?

SAM: So, an ICO is a more modern one. And it’s kind of fraught with some concerns around regulations and transparency today. But I think since Thanksgiving there’s been a massive wave of conversation about cryptocurrencies. And an ICO is essentially a way of creating your own cryptocurrency. The way I always explain to people, I love the analogy that people make around, think of it like I want to go build an amusement park. And in that amusement park, everything, rides, food, everything, is going to be denominated and payable in Sam-bucks.

CHARLES: Ah, right.

SAM: And… [Chuckles] And so, my options…

CHARLES: [Laughs] That makes sense.

SAM: Yeah. And my options are I can go to a bank and borrow money, I can go to investors and say, “Hey, give me the 10 million dollars it’s going to take to build it,” or I can just go to the people in the place where I’m building it and say, “You want this amusement park to exist? Why don’t you pre-buy these Sam-bucks?” And each one is going to cost a dollar today. And we create this universe of Sam-bucks and they’re essentially valuable once you can use them in the park. And there are certainly exceptions. There are other versions of cryptocurrencies and other uses for them. But that’s a conversation for another day.

CHARLES: Ah, mm.

SAM: I think that’s just a good, easy way to understand it.

CHARLES: Oh no, I like that. It’s like, well not quite like carnival tickets. But yeah, that’s something that everyone’s familiar with. Same thing as the Xbox Marketplace. Very similar thing. So, the idea is you would buy a bunch of Sam-bucks… you would get them at pennies on the dollar, so to speak, today.

SAM: Yeah, right. By the time it opens, maybe a hotdog would cost just one Sam-buck.

CHARLES: Right.

SAM: Whereas, when it’s coming in, we’d have to spend five dollars to get that one Sam-buck. Right, the idea being those people who got in early will be rewarded. And you can see it’s like a further extension of a Kickstarter or something else that you’re allowing people to pre-buy into a network.

CHARLES: Right. Right, okay. I can see that.

ERICK: That’s very interesting. [Laughs]

CHARLES: And so, it’s got a range of options too, because if you’re really interested in the services you can go ahead and spend them on the services and get a lot of value that way or you can actually trade for someone who does want the services if you don’t.

SAM: I think that’s exactly right. And it’s just, the one that I think I would just caveat is there is a huge amount of concern at the moment, and maybe concern is too strong a word, but uncertainty around one, what are the value of these coins, these tokens? And two, how will governments react to something that looks potentially like a security or a currency? And so, that’s something that still is being worked through. And even though they haven’t figured that out there’s still a massive amount of money being raised through these ICOs.

CHARLES: [Laughs] So, it does beg the question. Why is a cryptocurrency necessary? Why not just use Xbox Marketplace points? Why not just say, “Here are Sam-bucks.”

SAM: [Chuckles]

CHARLES: And there’s a row in my database.

[Laughter]

CHARLES: That’s your balance of Sam-bucks.

SAM: So, I think we’re about to get way beyond the [inaudible]

[Laughter]

SAM: But I think the argument would be that some of these things are better decentralized. So in my example, you’re right. That might just make more sense. But I think there are some examples around cryptocurrencies that are supporting a network of decentralized services where a centralized database historically was inconvenient or didn’t provide the amount of transparency that people were looking for.

CHARLES: Right, right.

SAM: And so, that’s a topic for a whole other podcast.

CHARLES: Yeah, right. No, it makes sense.

SAM: [Laughs]

CHARLES: I think it’s a matter of scale, right? If you’re going to be just buying services but if you’re going to have secondary markets where you’re trading in this currency, I can see that. So, let’s… [Chuckles] We’ll reel that back in.

SAM: [Chuckles]

CHARLES: And ask a question that occurred to me. So now, we talked about your day-to-day. What exactly, when you’re looking at a company to basically give money to, what are you looking for? What are the things you’re like, “Oh man, I want to throw dollars at this company,” versus, “Mm. I’m going to keep them and give them some feedback and send them on their way.”

SAM: There’s always a set of factors that we evaluate. And I think the waiting is probably different for different types of investors. And then there’s I’d say for me as a corporate VC being a part of GE, there’s an extra lens which is, how is this relevant to GE? What does it mean for GE to be an investor? But if I think about just the kind of general industry lines it’s: team is a really big one. So, who’s building this company? Do I believe in their ability to reach this vision that they’re laying out for me? Another one would be technology. What have they actually built? Is that hard to build? Do the things they want to build in the future, will those be hard to build? And do they have the skills and the people to do it? Then their technology, maybe an extension of that would be intellectual property. And besides intellectual property, just defensibility of a business in general. So then, you start thinking about, can somebody else just come along and to the same thing? Because if so, then maybe there’s not a strong advantage in what the company has done so far. And then lastly, it’s also just traction. How far along are they? How much have they proven the ability to execute on the plan that they’re laying out?

CHARLES: Right.

ERICK: So, you’re a corporate investor. So, there’s other types of investor like an institutional VC? What are the differences between an institutional VC and a corporate VC and the other types of VC? Potentially what they’d be looking for, in terms of what they wanted best.

SAM: Yeah. So, I think generally I categorize investors as institutional or corporate. And corporate [inaudible]…

ERICK: Yeah.

SAM: Corporate or strategic. And then there are people who exist on a spectrum there. But generally, an institutional means this is a group that is raising money from a set of limited partners who are the people who invest in the fund that are pension funds or wealthy individuals. They’re large pools of institutional capital and their pure purpose is to earn return. And they may have a certain focus because they believe in this part of the market, or they like this kind of company or the stage of company. But essentially, their job is to return more money to the limited partners of that fund that were put in. That’s their role in the world.

CHARLES: What I’m hearing is that you want to invest… I guess the thing is you can experience return that’s not just cash. It’s not just dollars. You’ll experience return in raising the ocean of the business that GE is in, right? So…

SAM: You said it much better than I did.

[Laughter]

CHARLES: Well, it’s all… paraphrasing is actually easy.

[Laughter]

ERICK: Oh, yeah.

SAM: An important skill.

CHARLES: That makes a lot of sense. So, the question I have then is, you said you were looking for companies that kind of swim in a specific ocean. And each company is farther along. Are you usually finding this company I want to work with, like you are going out and finding them? Or they’re coming to you looking for investment? Or is it really just, depends.

SAM: So, we call that part of the process sourcing, sourcing investments. And they come from all over. So for us, there are a few different ways. One is we tend to be thesis-driven. Meaning we go out and we say, the world is changing in this way and therefore we’re interested in this kind of company. And so, we’ll proactively go out and research. We’re also, I mentioned, a little later stage. So, I don’t tend to do seed investments. I tend to do ‘Series A’ and more often ‘Series B’ and later. So, companies that have often already raised a seed round or raised a ‘Series A’ round. So, I can actually search databases to say, “Okay, in the last two years who has raised a seed round or ‘Series A’ round and these other things I’m looking for whether it be location or tied to investors or other things.” So, that’s one way of being proactive is saying I want to go out and look for companies in this space that look like this. And that can be either like I mentioned, desktop research like searching the web, searching databases. Or it can be just going to conferences, right?

CHARLES: Are you investing with a mind that eventually GE might acquire this company and integrate it into GE itself? Or is it really just, “Hey, we’re just going to take a part of it. We’re going to have maybe a seat on the board to be able to steer a little bit. But we’re pretty much going to let it be its own thing with its own autonomy and go where it was and just benefit through those secondary and tertiary effects.”

SAM: Yeah, acquisitions from our portfolio by GE happened. But they’re certainly not the explicit goal or our focus. I know we’ve had one, maybe two of our portfolio companies acquired by GE, one that I was directly working with called Bit Stew. So, we made the investment in the company. It was with the goal of using their data management platform for a lot of our applications. And at some point in working with GE and GE Digital, they decided, you know, this would make sense to be a part of GE. That wasn’t why we made the investment. But it did end up being acquired by GE. And I know the team is doing really well. And it’s been at GE for about a year now. So, it does happen. But when I said one or two, that’s versus a portfolio of a hundred plus companies.

CHARLES: Right.

SAM: Since we started investing. And so, that’s not what we’re looking to do every time. Much more often it’s about again, how does the company make GE more competitive and a better company, a better place to work. And then how do we help them advance their goals? Whether it be bringing them developers, or finding them other routes to market, or just being a customer.

CHARLES: Right.

SAM: So, that’s really how we think about strategic value. There’s a lot of different ways to create it.

CHARLES: Yeah, I’m curious. Because it seems like also in a lot of these companies you’re investing in potential competitors. Extensively you’re operating if not in the exact same market, maybe very similar markets. There’s a little bit of overlap. And so, you’re kind of investing in potential competitors, right? So, where’s the balance of here we’re funding our competitors versus we’re going to move into these markets ourselves.

SAM: Yeah, and funding of “competitors” can happen. I think that we talk about that more in theory and say, “Oh sure, we’d be willing to fund a company that’s out disrupting the space that we’re playing in.” And we do that. It’s rare that you see startups that are directly head-on competing with much more established companies like GE or other industrials or even other consumer companies. They don’t take these companies head-on because that’s not a way that startups have been successful in the past, right? We talk much more about disruption and saying, how is this company doing something that may indirectly compete with GE? So, you think about things like, for anybody that’s not familiar with GE… actually, a lot of people associate us with our appliances which we actually don’t manufacture anymore. That’s [inaudible].

[Chuckles]

SAM: We sold that business a few years ago. Almost everything we sell is like big, heavy industrial equipment. So, we sell aircraft engines, locomotives. We sell gas turbines, wind turbines. So, here and there a couple of things that do power generation. One trend that’s affecting that industry is distributed generation of energy, energy storage. And those are parts of the market that are a less significant part of GE’s business than say, heavy-duty gas turbines that sit in a power plant and generate a massive amount of power. And so, if you look at that and say, “Wow, GE Ventures is out funding storage companies. Does that mean they’re funding competitors?” Well, it means that we’re funding innovation that may disrupt the future of our business, but that’s part of being a VC and that’s part of the value that GE Ventures brings to GE.

CHARLES: Right.

SAM: We’re out there looking at markets before they’re large enough or in scope for GE.

CHARLES: Mmhmm, right. And so, yes you’re disrupting the space but then you’re going to be a part of that disruption and have strong connections to those markets if you need to actually migrate your business completely over to them. That’s kind of what I’m hearing.

SAM: Yeah, absolutely. Better to disrupt yourself, right?

CHARLES: [Chuckles]

SAM: And be a part of the ecosystem in the future because I think the future happens with or without you. And it’s really key that we get out in front of it and a part of that, a part of that discussion, a part of that process.

CHARLES: And so now, you’ve been saying that this is, GE, this has been pretty explosive? There’s a lot more happening through GE Ventures. There’s a lot more happening in other companies globally, having these corporate ventures. Where do you think the balance is going to lie to say, “Hey,” I’m just going to throw out some numbers, just for theory here, it’s like, “10% of our business is essentially this distributed network of semi-autonomous or mostly autonomous startups. And then we have our core business.” Does that stabilize at 50/50? Does it stabilize at 75% the other way with GE essentially becoming a capital management company? Or is it somewhere in the middle?

SAM: So, GE Ventures will never be a meaningful part of GE’s revenue, a meaningful part of its business as a percentage. The overall venture industry is full of funds that are on the order of like, bigger funds are on the order of, in the billions. The single-digit billions. And GE itself is a much, much larger company. Well over a hundred billion dollars in enterprise value. So, I think GE Ventures will always be a small part of the company financially. And the impact will be largely felt through how we help the rest of GE navigate the future.

ERICK: You said that sometimes you go and look for companies, startups to invest in or sometimes startups come to you or come to a VC looking for funding. Now, I’m a developer or a startup founder. And I’m going to look for funding. What are some of the mistakes or pitfalls that you see that startup founders or people with an idea fall into when looking for funding that you can help them avoid?

SAM: Yeah, and we do see companies that come to us. So, I mentioned a lot about how I go out looking for companies based on a thesis or a set of relevant factors or relevant things for GE. But we do have a number of inbound requests. People know some of the bigger VC brands. They know GE the big company. So, we do get inbound interest and we also get referrals from networks of VCs and some are employees and other things. But for the companies that are seeking us out, the ones that are going out looking for funding, there are some things that are really well-known in Silicon Valley and other places, or you could research online and find, but may not be obvious at first.

[Laughter]

SAM: I think that’s one thing that I see, too. You have some company that comes in and say, “Look, here are the parts I’ve figured out and here are the parts I still have to figure out.” And that’s a really good conversation to have. There are other companies where they say, “Look, we’ve figured the whole thing out. We just want you to give us some money.” And I don’t think a lot of investors necessarily buy into that. And certainly, there are investors of every stripe. So, I may be speaking too broadly. But I think that’s a really important part of the venture investment process, right? You’re looking not just for money but also for counsel and for somebody that you’re going to work with over the next, sometimes seven years or longer.

CHARLES: Yeah.

SAM: [Inaudible] going to be on your board and participating. So, it’s a really important part.

CHARLES: So, you’re looking, you’re actually looking not necessarily for all the answers but you’re looking for the questions that they’re asking, too.

SAM: Yeah, absolutely. And demonstrating they understand the ins and outs of the business. And that they have the capacity to carry this onto that next stage and hopefully beyond.

CHARLES: Mmhmm. So, now you said something that caught my interest there that you work with some people sometimes seven years. You enter into these long relationships. Do you generally ever do any type of, I want to say almost like… mentoring might be too strong of a word, but in the pre-investment, in other words before you actually invest in a company, do you ever work with them to prepare them for investment to say, “Hey, I think there’s potential here. Work on A, B, and C and then let’s talk.” And you have this image in your mind. You go, you pitch to an investor, and it’s either thumbs up or it’s like thumbs down and you never talk to them again. Versus, is there some ground in between where there’s a conversation that evolves that eventually ends up in an investment being made?

SAM: Absolutely. I think one of the parts of this industry is even when I’m not an investor in a company, I may know a company and say, “It’s not a fit for me for GE Ventures but I still think that we can provide help.” It’s one of the things I love about tech and about venture in general, is that people are often willing to pitch in, even when they don’t have a direct financial incentive. And so, I see that a lot whether it’s helping a company where we’ve met them and we later see an opportunity and say, “Oh, you should go and talk to this company or that company.”

ERLICK: You said something, metrics. So, a venture capitalist, after they make an investment, what are some of the expectations that they may hold this startup that they just invested in… what are those expectations that they may hold them accountable for? Or those metrics that they’ll be looking at?

SAM: Yeah, so I think some of the really high-level ones that are common across businesses, generally growth is a really big one. So, I almost said revenue. But I wanted to caveat…

[Laughter]

SAM: And say growth could mean different things. It could mean number of developers. It could mean number of downloads if you’re an app. It depends on what the business is. But I think growth is a huge one. Growth is a really important, that top line, that’s what’s going to drive a lot of the value in the business. And then below that, demonstrating that you can hit the milestones around things like margins. So, how profitable is each unit you’re selling? Or how profitable is each customer? And lastly, how are you doing managing your spend? So, that’s great that you’re earning the right amount of money for each customer, but are you doing it by… do you have a massive number of employees and offices and all the things that are too expensive to allow you to use your money wisely as you reach the next stage? And so, those are the big milestones. It’s really just growth, margins, and operating cost or burn rate as we call it.

CHARLES: Mmhmm. So, that sounds like a lot of work to actually evaluate these companies. Do you do your due diligence once you’ve already moved in pretty solidly into the process?

SAM: Yeah, these processes can move really fast. And depending on the timing, generally it’s, you jump in, you learn as much as you can, as fast as you can, and you make a decision so the company can move on. I’ll say there’s a lot of work that goes into considering and deciding which companies to spend more time on, both for us and for them. We don’t want to waste a company’s time evaluating, going through more meetings, if it’s not a really strong candidate for us. Because they could be spending that time better with other investors who are a better fit.

ERLICK: You said that it was growth, spend, and profits were some of the metrics. That is almost all of the essential components of a business plan. I remember one time, one of our previous conversations, you emphasized how important it was for companies, or even at just a simple startup, to put together a basic business plan. Is that something that you can elaborate on a little?

SAM: Yeah. So, most companies show up with a pitch deck. So, they have a set of PowerPoint slides and then they have a set of materials behind that where if you go deeper into an area they may have a white paper about their technology and they may have an Excel financial model that explains why they have these expectations about what growth and margins and all those things will look like. So, there are all of those pieces that come together into a business plan. The business plan could be written or it could be that PowerPoint. But very traditionally, it’s a PowerPoint or some kind of presentation that is shared in person. There’s usually a version that’s sent in advance to confirm that the company and the investors should meet.

ERLICK: Yeah, because I see… well, I know a few people that have startup ideas and they kind of put the business plan on the back burner and put the actual prototype more at the forefront. They say, “Oh, we can worry about the business plan later.” [Laughs]

SAM: [Chuckles] Well, I think… there’s something to be said to that. There’s something to be said for product and growth winning. So if you… Let’s start at the early stages. If you have something that’s working and that’s really obvious, you may not need a…

ERLICK: True.

SAM: To go raise money. It all comes down to, do you have enough to get enough investors interested to raise the round that you want to raise? Because you want to have enough investors involved, enough demand, that you can be selective about who you want to work with and on what terms, right? So, what valuation and how much of the company am I giving them, and all of those things. So, if you can do all of those things with nothing but an app and one chart that shows a hockey stick of growth, that’s awesome.

CHARLES: You’re hot.

[Laughter]

SAM: Often it does require much more and a much longer plan. So, even if you say, “Look, it’s growing like crazy,” there’s usually some set of questions behind that. So, that’s great. Your free app is growing like crazy. How are you going…?

[Laughter]

SAM: To get paid for that? And you’ll talk about that. And you’ll say, “Here are the things we’re planning on doing,” or here are the assumptions that we’re making. And the more original, the more unique the business model is, the more discussion and explanation that may require. And that’s where the business plan and a pitch deck come in handy, because it’s a really good presentation aide or pre-reading to get to that answer faster.

ERLICK: So, this evaluation it seems, is a two-way street. The VCs evaluating the company and also the company or the startup evaluating the VC to know whether it’s going to be a good relationship.

SAM: Oh, absolutely. Yeah, the best companies have choice. They have a number of investors who are interested in funding them. And certainly, that might be different at different stages or at different times, depending on what’s going on in the economy and in tech and in other places. But generally, VC is a very competitive industry. I’m trying to sell my money and services as an investor versus other options that you have. And so, while it’s maybe not as competitive as only one of us can buy the company like in an M&A situation. There are often more than one investor. There’s still a very intense set of competition around, okay, who’s going to be involved in the deal? How much money will they be able to invest? So, that’s something that really can come in handy for founders.

ERLICK: And what was that you just said there? M&A situation?

SAM: Oh, sorry. When a company is being bought. So, when a company is being bought, it can look kind of like a fundraising process, but instead of selling a part of a company, you’re selling the whole thing. And so, in that case, obviously it’s a competitive situation where there’s only one winner. And this is a different process. Often, the rounds that we’re a part of, we’re not… we’re buying a minority stake just like any VC. We may be buying 5% of a company, 10% of a company. And often we’re being joined by other venture investors. We really actually commonly partner with the institutional firms and they’ll take a board seat. We’ll invest alongside them and be an observer on the board and provide counsel. And so, it is a very competitive process. And that, while M&A is a winner-take-all, there is one buyer who is ultimately going to own this company going forward, the investing process for a venture is much more collaborative. But it is still competitive, because there can only be so many investors in one company.

CHARLES: And you want to choose the right one on both… the right set. Alright. Well, I think we’re running up against time. This has been a fascinating conversation into an aspect of our industry that really is providing the fuel that drives so much of this forward. So, I guess I’ll close by asking you, already talked about Resin. We had them on the podcast. We love them. Are there any conferences or products that you’re investing in that you feel like our audience might want to know about or anything like that?

SAM: Well one, you mentioned Resin.

CHARLES: Yeah.

SAM: I know you guys have been a good friend to them and Elrick and I met at their hackathon. I would recommend to anybody, go try it out. It’s a really cool way to play with hardware products. I am not a developer and I required a lot of help from Elrick at the hackathon.

[Laughter]

SAM: But at the same time, it is something that almost anybody can pull out of a box and start playing with. So, I think that’s a great one. The episode you did on them were fantastic. So, I really enjoyed those ones. I’d say in general, I’m always out looking to meet new companies that are going to benefit from working with GE. I spend a lot of my time not just trying to invest but also trying to find partnerships for companies that we’re looking at within GE, either selling to us or working with us. And so, if somebody thinks that there’s an opportunity to do that, then I encourage them to reach out. Because I think there’s a ton of opportunity. It’s a really big company that really has a ton of opportunity for other partners.

CHARLES: Alright. If they wanted to reach out, how would they get in touch with you?

SAM: Yeah, I think maybe the best way to initially make contact, I tend to be pretty active on Twitter. So, my handle is just @SamCates. S-A-M-C-A-T-E-S. And you can also learn more through our website. If you’re curious about some of the businesses I mentioned, so just GEVentures.com. And it’s about to go through a whole refresh. So, go check it out.

CHARLES: Alright. Well, fantastic. We will definitely look for that. And for everybody else, you can get in touch with us on Twitter at @TheFrontside or send us a line at [email protected]. Thank you everybody for listening. Thank so, so much Sam, for being on the podcast.

SAM: Yeah, of course. It was a blast. I’m a big podcast fan and I’ve really enjoyed catching up on your episodes.

CHARLES: Ah, and thank you Elrick, always.

ERICK: It was great.

SAM: Elrick, when you finish building your Raspberry Pi Battleships, I want to play.

CHARLES: [Laughs]

ERICK: Oh, yes. Yes. It’s in the works, man. It’s in progress.

SAM: Alright, I’m waiting.

CHARLES: Alright. Well, take it easy, everybody.

Tracy Lee: @ladyleet | ladyleet.com

Ben Lesh: @benlesh | medium.com/@benlesh

Show Notes:

Resources:

Transcript:

CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 91. My name is Charles Lowell, a developer here at The Frontside and your podcast host-in-training. Joining me today on the podcast is Elrick Ryan. Hello, Elrick.

ELRICK: Hey, what’s up?

CHARLES: Not much. How are you doing?

ELRICK: I’m great. Very excited to have these two folks on the podcast today. I feel like I know them…

CHARLES: [Laughs]

ELRICK: Very well, from Twitter.

CHARLES: I feel like I know them well from Twitter, too.

ELRICK: [Laughs]

CHARLES: But I also feel like this is a fantastic company that is doing a lot of great stuff.

ELRICK: Yup.

CHARLES: Also not in Twitter. It should be pointed out. We have with us Tracy Lee and Ben Lesh from This Dot company.

TRACY: Hey.

CHARLES: So first of all, why don’t we start, for those who don’t know, what exactly is This Dot? What is it that you all do and what are you hoping to accomplish?

TRACY: This Dot was created about a year ago. And it was founded by myself and Taras who work on it full-time. And we have amazing people like Ben, who’s also one of our co-founders, and really amazing mentors. A lot of our friends, when they refer to what we actually do, they like to call it celebrity consulting.

[Laughter]

TRACY: Which I think is hilarious. But it’s basically core contributors of different frameworks and libraries who work with us and lend their time to mentor and consult with different companies. So, I think the beautiful part about what we’re trying to do is bring together the web. And we sort of do that as well not only through consulting and trying to help people succeed, but also through This Dot Media where it’s basically a big playground of JavaScripting all the things. Ben and I do Modern Web podcast together. We do RX Workshop which is RxJS training together. And Ben also has a full-time job at Google.

CHARLES: What do they got you doing over there at Google?

BEN: Well, I work on a project called Alkali which is an internal platform as a service built on top of Angular. That’s my day job.

CHARLES: So, you’ve been actually involved in all the major front-end frameworks, right, at some point?

BEN: Yeah, yes. I got my start with Angular 1 or AngularJS now, when I was working as a web developer in Pittsburgh, Pennsylvania at a company called Aesynt which was formerly McKesson Automation. And then I was noticed by Netflix who was starting to do some Angular 1 work and they hired me to come help them. And then they decided to do Ember which is fine. And I worked on a large Ember app there. Then I worked on a couple of large React apps at Netflix. And now I’m at Google building Angular apps.

CHARLES: Alright.

BEN: Which is Angular 5 now, I believe.

CHARLES: So, you’ve come the full circle.

BEN: Yeah. Yeah, definitely.

CHARLES: [Chuckles] I have to imagine Angular’s changed a lot since you were working on it the first time.

BEN: Yeah. It was completely rewritten.

TRACY: I feel like Angular’s the new Ember.

CHARLES: Angular is the new Ember?

TRACY: [Laughs]

BEN: You think?

TRACY: Angular is the new Ember and Vue is the new AngularJS, is basically. [Laughs]

CHARLES: Okay.

[Laughter]

CHARLES: What’s the new React then?

BEN: Preact would be the React.

CHARLES: Preact? Okay, or is Glimmer…

BEN: [Laughs] I’m just…

CHARLES: Is Glimmer the new React?

BEN: Oh, sure. [Laughs]

CHARLES: It’s important to keep these things straight in your head.

BEN: Yeah, yeah.

CHARLES: Saves on confusion.

TRACY: Which came first?

[Chuckles]

BEN: Too late. I’m already confused.

CHARLES: So now, before the show you were saying that you had just, literally just released RxJS, was it 5.5.4?

BEN: That’s right. That’s right. The patch release, yeah.

CHARLES: Okay. Am I also correct in understanding that RxJS has kind of come to very front and center position in Angular? Like they’ve built large portions of framework around it?

BEN: Yeah, it’s the only dependency for Angular. It is being used in a lot of official space for Angular. For example, Angular Material’s Data Table uses observables which are coming from RxJS. They’ve got reactive forms. The router makes use of Observable. So, the integration started kind of small which HTTPClient being written around Observable. And it’s grown from there as people seem to be grabbing on and enjoying more the React programming side of things. So, it’s definitely the one framework that’s really embraced reactive programming outside of say, Cycle.js or something like that.

CHARLES: Mmhmm. So, just to give a general background, how would you characterize RxJS?

BEN: It’s a library built around Observable. And Observable is a push-based primitive that gives you sets of events, really.

CHARLES: Mmhmm.

BEN: So, that’s like Lodash for events would be a good way to put it. You can take anything that you can get pushed at you, which is pretty much value type you can imagine, and wrap it in an observable and have it pushed out of the observable. And from there, you have a set of things that you can combine. And you can concatenate them, you can filter them, you can transform them, you can combine them with other sets, and so on. So, you’ve got this ability to query and manipulate in a declarative way, events.

CHARLES: Now, Observable is also… So, when Jay was on the podcast we were talking about Redux observable. But there was outside of the context of RxJS, it was just observables were this standalone entity. But I understand that they actually came from the RxJS project. That was the progenitor of observables even though there’s talk of maybe making them part of the JavaScript spec.

BEN: Yeah, that’s right. That’s right. So, RxJS as it stands is a reference implementation for what could land in JavaScript or what could even land in the DOM as far as an observable type. Observable itself is very primitive but RxJS has a lot of operators and optimizations and things written around Observable. That’s the entire purpose of the library.

CHARLES: Mmhmm. So, what kind of value-adds does it provide on top of Observable? If Observable was the primitive, what are the combinators, so to speak?

BEN: Oh, right. So, similar to what Lodash would add on top of say, an iterable or arrays, you would have the same sorts of things and more inside of RxJS. So, you’ve got zip which you would maybe have seen in Lodash or different means of combines. Of course, map and ‘merge map’ which is like a flattening sort of operation. You can concatenate them together. But you also have these time-based things. You can do debouncing or throttling of events as they’re coming over in observable and you create a new observable of that. So, the value-add is the ability to compose these primitive actions. You can take on an observable and make a new observable. We call it operators. And you can use those operators to build pretty much anything you can imagine as far as an app would go.

CHARLES: So, do you find that most of the time all of the operators are contained right there inside RxJS? Or if you’re going to be doing reactive programming, one of your tasks is going to be defining your own operators?

BEN: No, pretty much everything you’d need will be defined within RxJS. There’s 60 operators or so.

CHARLES: Whoa, that’s a lot.

BEN: It’s unlikely that someone’s going to come up with one. And in fact, I would say the majority of those, probably 75% of those, you can create from the other 25%. So, some of the much more primitive operators could be used…

TRACY: Which is sort of what Ben did in this last release, RxJS 5…. I don’t know remember when you introduced the lettable operators but you…

BEN: Yeah, 5.5.

TRACY: Implemented [inaudible] operators.

BEN: Yeah, so a good portion of them I started implementing in terms of other operators.

CHARLES: Right. So, what was that? I didn’t quite catch that, Tracy. You said that, what was the operator that was introduced?

TRACY: So, in one of the latest releases of RxJS, one of the more significant releases where pipeable operators were introduced, what Ben did was he went ahead and implemented a lot of operators that were currently in the library in terms of other operators, which was able to give way to reduce the size of the library from, I think it was what, 30KB bundled, gzipped, and minified, to about 30KB, which was about 60 to 70% of the operators. Right, Ben?

BEN: Yeah. So, the size reduction was in part that there’s a lot of factors that went into the size reduction. It would be kind of hard to pin it down to a specific operator. But I know that some of the operators like the individual operators themselves, by reimplementing reduce which is the same as doing as scan and then take last, implementing it in terms of that is going to reduce the size of it probably 90% of that one particular file. So, there’s a variety of things like that that have already started and that we’re going to continue to do.

CHARLES: Mmhmm.

TRACY: And I think another really interesting thing is a lot of people when learning RxJS, they… it’s funny because we just gave an RX Workshop course this past weekend and the people that were there just were like, “Oh, we’ve heard of RxJS. We think it’s a cool new thing. We have no plans to implement it in real life but let’s just play around with it and let me learn it.” I think as people are starting to learn RxJS, one of the things that gets them really overwhelmed is this whole idea that they’re having to learn a completely new language on top of JavaScript or what operators to use. And one of our friends, Brian Troncone who is on the Learning Team, the RxJS Learning Team, he pulled up the top 15 operators that were most commonly searched on his site. And some of them were ‘switch map’, ‘merge map’, ‘fork join’, merge, et cetera. So, you can sort of tell that even though the library has quite a few… it’s funny because Ben, I think the last RX Workshop you were using pairs and you had never used it before.

BEN: Yeah.

TRACY: So, it’s always amusing for me how many people can be on the core team but have never implemented RxJS…

CHARLES: [Laughs]

TRACY: A certain way.

BEN: Right. Right, right, right.

CHARLES: You had said one of the recent releases was about making it more friendly for functional programming. Is that a subject that we can explore? Because using observables is already pretty FP-like.

BEN: What it was before is we had dot chaining. So, you would do ‘dot map’ and then call a method and then you get an observable back. And then you’d say ‘dot merge’ and then you’d call a method on that, and so on and so forth. Now what you have is kind of a Ramda JS style pipe function that just takes a comma-separated list of other functions that are going to act upon the observable. So, it reads pretty much the same with a little more ceremony around it I guess. But the upside is that you can develop your operators as just higher-order functions.

CHARLES: Right. And you don’t have to do any monkey-patching of prototypes.

BEN: Exactly, exactly.

CHARLES: Because actually, okay, I see. This is actually pretty exciting, I think. Because we actually ran into this problem when we were using Redux Observable where we wanted to use some operators that were used by some library but we had to basically make a pull request upstream, or fork the upstream library to include the operators so that we could use them in our application. It was really weird.

BEN: Yeah.

CHARLES: The reason was because it was extending the observable prototype.

BEN: Yeah. And there’s so many… and that’s one way to add that, is you extend the observable prototype and then you override lift so you return the same type of observable everywhere. And there are so many things that lettable operators solved for us. For example…

CHARLES: So, lettable operators. So, that’s the word that Tracy used and you just used it. What are lettable operators?

BEN: Well, I’ve been trying to say pipeable and get that going instead of lettable. But basically there’s an operator on RxJS that’s been there forever called let. And let is an operator and what you do is you give it a function. And the function gives you the source observable and you’re expected to return a new observable. And the idea is that you can then write a function elsewhere that you can then compose in as though it were an operator, anywhere you want, along with your other dot-chained operators. And the realization I had a few months ago was, “Well, why don’t we just make all operators like this?” And then we can use functional programming to compose them with like a reduce or whatever. And that’s exactly what the lettable operators are. And that’s why I started calling them lettable operators. And I kind of regret it now, because so many people are saying it and it confuses new people. Because what in the world does lettable even mean?

CHARLES: Right. [Laughs]

BEN: So, they are pipeable operators or functional operators. But the point is that you have a higher-order function that returns a function of a specific shape. And that function shape is, it’s a function that receives an observable and returns an observable, and that’s it. So, basically it’s a function that transforms an observable into a new observable. That’s all an operator. That’s all an operator’s ever been. It’s just this is in a different flavor.

CHARLES: Now, I’m curious. Why does it do an observable into an observable and not a stream item into an observable? Because when you’re actually chaining these things together, like with a map or with a ‘flat map’ or all these things, you’re actually getting an individual item and then returning an observable. Well, I guess in this case of a map you’re getting an item and returning an item. But like…

BEN: Right, but that’s not what the entire operation is. So, you’ve got an operation you’re performing whenever you say, if you’re to just even dot-chain it, you’d say ‘observable dot map’. And when you say ‘dot map’, it returns a new observable. And then you say ‘dot filter’ and it returns another new observable.

CHARLES: Oh, gotcha, gotcha, gotcha. Okay, yeah, yeah, yeah. Yeah, yeah.

BEN: So, this function just embodies that step.

CHARLES: I see, I see. And isn’t there some special… I feel like there’s some proposal for some special JavaScript syntax to make this type of chaining?

BEN: Yeah, yeah, the pipeline operator.

CHARLES: Okay.

BEN: I don’t know. I think that’s still at stage one. I don’t know that it’s got a lot of headway. My sources and friends that are in the TC39 seem to think that it doesn’t have a lot of headway. But I really think it’s important. Because if you look at… the problem is we’re using a language where the most common use case is you have to build it, get the size as small as possible because you need to send it over the wire to the browser. And understandably, browsers don’t want to implement every possible method they could on say, Array, right?

CHARLES: Mmhmm, right.

BEN: There’s a proposal in for ‘flat map’. They could add zip to Array. They could add all sorts of interesting things to Array just by itself. And that’s why Lodash exists, right?

CHARLES: Right.

BEN: Is because not everything is on Array. And then so, the onus is then put on the community to come up with these solutions and the community has to build libraries that have these constraints in size. And what stinks about that is then you have say, an older version of Lodash where you’d be like, “Okay, well it has 36 different functions in it and I’m only using 3 of them. And I have to ship them all to the browser.”

CHARLES: Mmhmm.

BEN: And that’s not what you want. So, then we have these other solutions around tree-shaking and this and that. And the real thing is what you want is you want to be able to compose things left to right and you want to be able to have these functions that you can use on a particular type in an ad hoc way. And there’s been two proposals to try to address this. One was the ‘function bind’ operator,

CHARLES: Mmhmm.

BEN: Which is colon colon. And what that did is it said, “You can use this function as a method, as though it were a method on an object. And we’ll make sure that the ‘this’ inside that function comes from the instance that’s on the left-hand side of colon colon.”

CHARLES: Right.

BEN: That had a bunch of other problems. Like there’s some real debate I guess on how they would tie that down to a specific type. So, that kind of fell dead in the water even though it had made some traction. And then the pipeline operator is different. And then what it says is, “Okay, whatever is on the…” And what it looks like is a pipe and a greater than right next to each other. And whatever’s on the left-hand side of that operand gets passed as the first argument to the function on the right-hand side of that operand.

CHARLES: Mmhmm.

BEN: And so, what that means is for the pipeable operators, instead of having to use a pipe method on observable, you can just say, “instance of observable, pipeline operator and an operator, and then pipeline operator, and then the Rx operator, and then pipeline operator and the Rx operator, and so on.” And it would just be built-in. And the reason I think that JavaScript really needs it is that means that libraries like Lodash can be written in terms of simple functions and shipped piece-meal to the browser exactly as you need them. And people would just use the pipeline operator to use them, instead of having to wrap something in a big object so you can dot-chain things together or come up with your own functional pipe thing like RxJS had to.

CHARLES: Right. Because it seems it happens again and again, right? Lodash, RxJS, jQuery. You just see this pattern of chaining, which is, you know…

BEN: Yeah, yeah. People want chaining. People want left to right composition.

CHARLES: Mmhmm.

BEN: And it’s problematic in a world where you want to shake off as much unused garbage as possible. And the only way to get dot chaining is by augmenting a prototype. There’s all sorts of weird problems that can come with that. And so, the functional programming approach is one method. But then people look at it and they say, “Ooh, yuck. I’ve got to wrap things in a function named pipe. Wouldn’t it be nicer if there was just some syntax to do this?” And yeah, it would be nicer. But I have less control over that.

CHARLES: Right. But the other alternative is to have right to left function composition.

BEN: Right, yeah.

CHARLES: But there’s not any special syntax for that, either.

BEN: Not very readable.

CHARLES: Yeah.

BEN: So, you just wrap everything. And the innermost call is the first one and then you wrap it in another function and you wrap that in another function, and so on. Yeah, that’s not [inaudible]. But I will say that the pipe function itself is pretty simple. It’s basically a function that takes a rest of arguments that are all functions.

CHARLES: Mmhmm.

BEN: And so, you have this array of functions and you just reduce over it and call them. Well, you return a function. So, it’s a higher function. You return a function that takes an argument then you reduce over the functions that came in as arguments and you call each one of them with whatever result was from the previous.

CHARLES: Right. Like Tracy mentioned in the pre-show, I’m an aspiring student of functional programming. So, would this be kind of like a monoid here where you’re mashing all these functions together? Is your empty value? I’m just going to throw it out there. I don’t know if it’s true or not, but that’s my conjecture.

BEN: Yes. Technically, it’s a monoid because it wouldn’t work unless it was a monoid. Because monoids, I believe the category theory I think for monoid is that monoids can be concatenated because they definitely have an end.

CHARLES: Right.

BEN: So, you would not be able to reduce over all those functions and build something with that, like that, unless it was a monoid. So yeah, the fact that there’s reduction involved is a cue that it’s a monoid.

CHARLES: Woohoo! Alright.

[Laughter]

CHARLES: Have you found yourself wanting to apply some of these more “rigorous” formalisms that you find out there in the development of RxJS or is that just really a secondary concern?

BEN: It’s a secondary concern. It’s not something that I like. It’s something I think about from time to time, when really, debating any kind of heavy issue, sometimes it’s helpful. But when it comes to teaching anybody anything, honestly the Haskell-isms and category theory names, all they do is just confuse people. And if you tell somebody something is a functor, they’re like, “What?” And if you just say it’s mappable, they’re like, “Oh, okay. I can map that.”

CHARLES: [Laughs] Right, right.

BEN: And then the purists would be like, “But they’re not the same thing.” And I would be like, “But the world doesn’t care. I’m sorry.”

CHARLES: Yeah, yeah. I’m kind of experiencing this debate myself. I’m not quite sure which side I fall on, because on the one hand it is arbitrary. Functor is a weird name. But I wish the concept of mappable existed. It does, but I feel like it would be handy if people… because there’s literally five things that are super handy, right? Like mappable, if we could have a name for monoid. But it’s like, really, you just need to think in terms of these five constructs for 99% of the stuff that you do. And so, I always wonder, where does that line lie? And how… mappable, is that really more accessible than functor? Or is that only because I was exposed to the concept of mapping for 10 years before I ever heard the F word.

BEN: Yes, and yes. I mean, that’s…

CHARLES: [Laughs]

BEN: Things that are more accessible are usually more accessible because of some pre-given knowledge, right? What works in JavaScript probably isn’t going to work in Haskell or Scala or something, right?

CHARLES: Mmhmm.

BEN: If someone’s a Java developer, certain idioms might not make sense to them that come from the JavaScript world.

CHARLES: Right. But if I was learning like a student, I would think mappable, I’d be thinking like, I would literally be thinking like Google Maps or something like that. I don’t know.

BEN: Right, right. I mean, look at C#. C#, a mapping function is always going to be called select, right, because that’s C#. That’s their idiom for the same thing.

CHARLES: Select?

BEN: Yeah.

CHARLES: Really?

BEN: Yeah, select. So, they’ll…

CHARLES: Which in Ruby is like find.

BEN: Yeah. there’s select and then, what’s the other one, ‘select many’ or something like that.

[Chuckles]

BEN: So, that’s C#.

CHARLES: Oh, like it’s select from SQL. Okay.

BEN: Yeah, I think that’s kind of where it came from because people had link and then they had link to SQL and then they’re like, well I want to do this with regular code, with just using some more… less nuanced expressions. So, I want to be able to do method calls and chain those together. And so, you end up with select functions. And I think that that exists even in Rx.NET, although I haven’t used Rx.NET.

CHARLES: Hmm, okay.

ELRICK: So, I know you do a lot of training with Rx. What are some of the concepts that people struggle with initially?

TRACY: I think when we’re teaching RX Workshop, a lot of the people sort of… I’ll even see senior level people struggle with explaining it, is the difference between observables and observers and then wrapping their head around the idea that, “Hey, observables are just functions in JavaScript.” So, they’re always thinking observables are going to do something for you. Actually, it’s not just in Angular but also in React, but whenever someone’s having issues with their Rx applications, it’s usually something that they’re like nesting observables or they’re not subscribing to something or they’ve sort of hot-messed themselves into a tangle. And I’m sure you’ve debugged a bunch of this stuff before. The first thing I always ask people is, “Have you subscribed?” Or maybe they’re using an Angular… they’re using pipes async but they’re also calling ‘dot subscribe’ on their observable.

BEN: Yeah. So, like in Angular they’ll do both. Yeah. There’s that. I think that, yeah, that relates to the problem of people not understanding that observables are really just functions. I keep saying that over and over again and people really don’t seem to take it to heart for whatever reason.

[Chuckles]

BEN: But you get an observable and when you’re chaining all those operators together, you’re making another observable or whatever, observables don’t do anything until you subscribe to them. They do nothing.

CHARLES: Shouldn’t they be called like subscribable?

BEN: Yes.

[Chuckles]

BEN: They probably should. But we do hand them an observer. So, you are observing something. But the point being is that they don’t do anything at all until you subscribe to them. And in that regard, they’re like functions, where functions don’t do anything unless you call them. So, what ends up happening with an observable is you subscribe to it. You give it an observer, three callbacks which are then coerced into an observer. And it takes that observer and it hands it to the body of this observable definition and literally has an observer inside of there. And then you basically execute that function synchronously and do things, whatever those things are, to set up some sort of observation. Maybe you spin up a WebSocket and tie into some events on it and call next on the observer to get values out of your observable. The point being that if you subscribe to an observable twice, it’s the same thing as calling a function twice. And for some reason, people have a hard time with that. They think, if I subscribe to the observable twice, I’ve only called the function once.

CHARLES: I experienced this confusion. And I remember the first time that that… like, I was playing with observables and the first time I actually discovered that, that it was actually calling my… now what do you call the function that you pass to the constructor that actually does, that calls next or that gets passed the observer?

TRACY: [Inaudible]

BEN: I like to call it an initialization function or something. But the official name from the TC39 proposal is subscriber function.

CHARLES: Subscriber function. So, like…

BEN: Yeah.

CHARLES: I definitely remember it was one of those [makes explosion sound] mind-blowing moments when I realized when I call my subscribe method, the entire observable got run from the very beginning. But my intuition was that this is an object. It’s got some shared state, like it’s this quasar that I’m now observing and I’m seeing the flashes of light coming off of it. But it’s still the same object. You think of it as having yeah, not as a function. Okay. No one ever described it to me as just a function. But I think I can see it now.

ELRICK: Yeah, me neither.

CHARLES: But yeah, you think of it in the same way that most people think of objects, as like, “I have this object. I have a reference to it.” Let observable equal new observable. It’s a single thing. It’s a single identity. And so, that’s the thing that I’m observing. It’s not that I’m invoking this observable to observe things. And I think that’s, yeah, that’s a subtle nuance there. I wish I had taken y’all’s course, I guess is what I’m saying.

ELRICK: Yeah.

BEN: Yeah. Well, I’ve done a few talks on it.

CHARLES: [Laughs]

BEN: I always try to tell people, “It’s just a function. It’s just a function.” I think what happens to a lot of people too is there’s the fact that it’s an object. But I think what it is, is people’s familiarity with promises does this. Because promises are always multicast. They are always “hot”. And the reason for this is because they’re eager. So, by the time you have a promise, whatever is producing value to the promise has already started. And that means that they’re inherently a multicast.

CHARLES: Right.

BEN: So, people are used to that behavior of, I can ‘then’ off of this promise and it always means one thing. And it’s like, yeah, because the one thing has nothing to do with the promise. It wasn’t [Chuckles]

CHARLES: Right.

BEN: This promise is just an interface for you to view something that happened in the past, where an observable is more low-level than that and more simple than that. It just states, “I’m a function that you call. I’m going to be able to do anything a function can do. And by the way, you’re giving me an observer and I’m going to do some stuff with that too and notify you via this observer that you handed me.” Because of that you could take an observable and close over something that had already started. Say you had a WebSocket that was already running. You could create a new observable and just like any function, close over that, externally create a WebSocket. And then everyone that subscribes to that observable is tying an observer to that same WebSocket. Then you’re multicast. Then you’re “hot”.

ELRICK: [Inaudible]

CHARLES: Right. So, I was going to say that’s the distinction that Jay was talking about. He was talking about we’re going to just talk about… he said at the very beginning, “We’re just going to talk about hot observable.”

ELRICK: Yup.

CHARLES: But even a hot observable is still theoretically evaluating every single time you subscribe. You’re getting a new observable. You’re evaluating that observable afresh each time. It just so happens that in the lexical scope of that observable subscriber function, there is this WebSocket?

BEN: Yeah. So, it’s the same thing. Imagine you wrote a function that when you called it created a new WebSocket and then… say, you wrote a new function that you gave an observer object to, right? An observer object has next, error, and complete. And in that function, when you called it, it created a new WebSocket and then it tied the ‘on message’ and ‘on close’ and whatever to your observer’s next method and your observer’s error message and so on. When you call that function, you would expect a new WebSocket to be created every single time.

CHARLES: Right. But really, it’s just a matter of scope.

BEN: Yeah. The thing people have a hard time with, with observables, is not realizing that they’re actually just functions.

CHARLES: Yeah. I just think that maybe… see, when I hear things like multicast and unicast, that makes me think of shared state, whereas when you say it’s just a matter of scope, well then I’m thinking more in terms of it being just a function. It just happens that this WebSocket was already [scoped].

BEN: Well, shared state is a matter of scope, right?

CHARLES: Yes, it is. It is. Oh, sorry. Shared state associated with some object identity, right?

BEN: Right.

CHARLES: But again, again, it’s just preconceptions, really. It’s just me thinking that I’ve had to manage lists of listeners and have multicast observers and single-cast observers and having to manage those lists and call notify on all of them. And that’s really not what’s happening at all.

BEN: Yeah. Well, I guess the real point is observables can have shared state or they could not have shared state. I think the most common version and the most composable version of them, they do not have any shared state. It’s just one of those things where just like a function can have shared state or it could be pure, right? There’s nothing wrong with either one of those two uses of a function. And there’s nothing wrong with either one of those two uses of Observable. So, honest to god, that is the biggest stumbling block I think that I see people have. That and if I had to characterize it I would say fear and loathing over the number of operators. People are like…

CHARLES: [Chuckles]

BEN: And they really think because everyone’s used to dealing with these frameworks where there’s an idiomatic way to do everything, they think there’s going to be an RxJS idiomatic way to do things. And that’s just patently false. That’s like saying there’s an idiomatic way to use functions. There’s not. Use it however it works. The end. It’s not…

CHARLES: Mmhmm, mmhmm.

BEN: You don’t have to use every operator in a specific way. You can use it however works for you and it’s fine.

ELRICK: I see that you guys are doing some fantastic work with your documentation. Was that part of RxJS 2.0 docs?

TRACY: I was trying to inspire people to take on the docs initiative because I think when I was starting to learn RxJS I would get really frustrated with the docs.

BEN: Yeah.

TRACY: I think the docs are greatly documented but at the same time if you’re not a senior developer who understands Rx already, then it’s not really helpful. Because it provides more of a reference point that the guys can go back and look at, or girls. So anyways, after many attempts of trying to get somebody to lead the project I just decided to lead the project myself.

[Laughter]

TRACY: And try to get… the community is interesting because I think because the docs can be sometimes confusing… Brian Troncone created LearnRxJS.io. There’s these other visualization projects like RxMarbles, RxViz, et cetera. And we just needed to stick everybody together. So, it’s been a project that I think has been going on for the past two months or so. We have… it’s just an Angular app so it’s probably one of the most easiest projects to contribute to.

BEN: I’m super pleased with all the people that have been working on that. Brian and everybody, especially on the accessibility front. Jen Luker [inaudible] came in and voluntarily… she’s like the stopgap for all accessibility to make sure everything is accessible before we release. So, that’s pretty exciting.

TRACY: Yeah.

ELRICK: Mmhmm.

TRACY: So funny because when me and Jen started talking, she was talking about something and then I was like, “Oh my god, I’m so excited about the docs.” She’s like, “I’m so excited, too! But I don’t really know why I’m excited. But you’re excited, so I’m excited. Why are you excited?”

[Laughter]

TRACY: I was like, “I don’t know. But I’m excited, too!”

[Chuckles]

TRACY: And then all of a sudden we have accessibility. [Laughs]

ELRICK: Mmhmm. Yeah, I saw some amazing screenshots. Has the new docs, have they been pushed up to the URL yet?

TRACY: Nah, they are about to. We were… we want to do one more accessibility run-through before we publish it. And then we’re going to document. We want to document the top 15 most viewed operators. But we should probably see that in the next two weeks or so, that the new docs will be… I mean, it’ll say “Beta, beta, beta” all over everything. But actually also, some of our friends, [Dmitri] from [Valas] Software, he is working on the translation portion to make it really easy for people to translate the docs.

CHARLES: Ah.

TRACY: So, a lot of that came from the inspiration from the Vue.js docs. we’re taking the versioning examples that Ember has done with their docs as inspiration to make sure that our versioning is really great. So, it’s great that we can lend upon all the other amazing ideas in the industry.

ELRICK: Oh, yeah.

CHARLES: Yeah, it’s fantastic. I can’t wait to see them.

ELRICK: Yeah, me neither. The screenshots look amazing. I was like, “Wow. These are some fabulous documentation that’s going to be coming out.” I can’t wait.

TRACY: Yeah. Thank you.

CHARLES: Setting the bar.

ELRICK: Really high.

[Laughter]

CHARLES: Actually, I’m curious. Because observables are so low-level, is there some use of them that… what’s the use of them that you found most surprising? Or, “Whoa, this was a crazy hack.”

BEN: The weirdest use of observables, there’s been quite a few odd ones. One of the ones that I did one time that is maybe in RxJS’s wheelhouse, it was just that RxJS already existed. So, I didn’t want to pull in another transducer library, was using RxJS as a transducer. Basically… in Netflix we had a situation where we had these huge, huge arrays of very large objects. And if you try to take something like that and then map it and then filter it and then map it and then filter it, we’re using Array map and filter, what ends up happening is you create all sorts of intermediary arrays in-memory. And then garbage collection has to come through and clean that up. And that locks your thread. And over time, we were experiencing slowness with this app. And it would just build up until eventually it ground to a halt.

CHARLES: So, will you just…

BEN: It saved garbage collection and it increased the performance of the app. But that’s just in an extreme case. I would never do that with just regular arrays. If anything, it was because it was huge, huge arrays of very large objects.

CHARLES: So, you would create an observable our of the array and then just feed each element into the observable one at a time?

BEN: Well, no. If you say ‘observable from’ and you give it an array, that’s basically what it does.

CHARLES: Okay.

BEN: It loops over the array and nexts those values out of the array synchronously.

CHARLES: I see, I see.

BEN: So, it’s like having a for loop and then inside of that for loop saying, “Apply the map. Apply the filter,” whatever, to each value as they’re going through. But when you look at it, if you had array map, filter, reduce, it’s literally just taking the first step and saying ‘observable from’ and wrapping that array and then the rest of it’s still the same.

CHARLES: Right. Yeah. No, that’s really cool.

BEN: That was a weirder use of it. I’ve heard tell of other things where people used observables to do audio synchronization, which is pretty interesting. Because you have to be very precise with audio synchronization. So, hooking into some of the Web Audio APIs and that sort of thing. That’s pretty interesting. The WebSocket multiplexing is something I did at Netflix that’s a little bit avant-garde for observable use because you essentially have an observable that is your WebSocket. And then you create another observable that closes over that observable and sends messages over the WebSocket for what you’re subscribed to and not subscribed to. And it enables you to very easily retry connections and these sorts of things. I did a whole talk on that. That one’s pretty weird.

CHARLES: Yeah. Man, I [inaudible] to see that.

BEN: But in the general use case, you click a button, you make an AJAX request, and then you get that back and maybe you make another AJAX request. Or like drag and drop and these sorts of things where you’re coordinating multiple events together, is the general use case. The non-weird use case for RxJS. Tracy does weird stuff with RxJS though.

[Laughter]

CHARLES: Yeah, what’s some weird uses of RxJS?

TRACY: I think my favorite thing to do right now is to figure out how many different IoT-related things I can make work with RxJS. So, how many random things can I connect to an application using that?

BEN: Tracy’s projects are the best. They’re so good.

[Laughter]

TRACY: Well, Ben and I created an application where you can take pictures of things using the Google Image API and it’ll spit back a set of puns for you. So, you take a picture of a banana, it’ll give you banana puns. Or you can talk to it using the speech recognition API. My latest thing is I really want to figure out how to… I haven’t figured out if Bluetooth Low Energy is actually enabled on Google Home Minis. But I want to get my Google Home Mini to say ‘booty’. [Inaudible]

[Laughter]

CHARLES: RxJS to the rescue.

[Laughter]

BEN: Oh, there was, you remember Ng-Cruise. We did Ng-Cruise and on there, Alex Castillo brought…

TRACY: Oh, that was so cool.

BEN: All sorts of interesting… you could read your brain waves. Or there was another one that was, what is it, the Microsoft, that band put around your wrist that would sense what direction your arm was in and whether or not your hand was flexed. And people…

TRACY: Yeah, so you could flip through things.

BEN: Yeah. And people were using reactive programming with that to do things like grab a ball on the screen. Or you could concentrate on an image and see if it went blurry or not.

ELRICK: Well, for like, Minority Report.

BEN: Oh, yeah, yeah. Literally, watching a machine read your mind with observables. That was pretty cool. That’s got to be the weirdest.

TRACY: Yeah, or we had somebody play the piano while they were wearing one of the brainwave… it’s called the OpenBCI project is what it is. And what you can do is you can actually get the instructions to 3D print out your own headset and then buy the technology that allows you to read brain waves. And so with that, it’s like… I mean, it was really awesome to watch her play the piano and just see how her brain waves were going super crazy. But there’s also these really cool… I don’t know if you guys have heard of Jewelbots, but they’re these programmable friendship bracelets that are just little Arduino devices that light up. I have two of them. I haven’t even opened them.

CHARLES: [Laughs]

TRACY: I’ve been waiting to play with them with you. I don’t know what we’re going to do, but I just want to send you lights. Flashing lights.

[Laughter]

TRACY: Morse code ask you questions about RxJS while you’re working.

[Laughter]

CHARLES: Yeah. Critical bug. Toot-toot-toot-too-too-too-too-toot-toot.

[Laughter]

CHARLES: RxJS Justice League.

TRACY: That would actually be really fun.

[Laughter]

TRACY: That would be really fun. I actually really want to do that. But…

CHARLES: I’m sure the next time we talk, you will have.

TRACY: [Laughs] Yes. Yes, yes, yes, I know. I know. we’ll do it soon. We just need to find some time while we’re not going crazy with conferences and stuff like that.

CHARLES: So, before we head out, is there any upcoming events, talks, releases, anything that we ought to be, we or the listeners, ought to be aware of?

TRACY: Yeah, so one of the things is that Ben and I this weekend actually just recorded the latest version of RX Workshop. So, if you want to learn all about the latest, latest, newest new, you can go ahead and take that course. We go through a lot of different things like multiplex WebSockets, building an application. Everywhere from the fundamentals to the more real world implementations of RxJS.

BEN: Yeah. Even in the fundamentals area, we’ve had friends of ours that are definitely seasoned Rx veterans come to the workshop. And most of them ask the most questions while talking about the fundamentals. Because I tend to dig into, either deep into the internals or into the why’s and how’s thing. Why and how things work. Even when it comes to how to subscribe to an observable. Deep detailed information about what happens if you don’t provide an error handler and certain cases and how that’s going to change in upcoming versions, and why that’s changing in upcoming versions, and what the TC39’s thoughts are on that, and so on and so forth. So, I try to get into some deeper stuff and we have a lot of fun. And we tend to be a little goofier at the workshops from time to time than we were in this podcast. Tracy and I get silly when we’re together.

TRACY: It’s very true.

[Laughter]

TRACY: But I think also, soon I think there are people that are going to be championing an Observable proposal on what [inaudible]. So, aside from the TC39 Observable proposal that’s currently still at stage one, I don’t know Ben if you want to talk a little bit about that.

BEN: Oh, yeah. So, I’ve been involved in conversations with folks from Netflix and Google as well, Chrome team and TC39 members, about getting the WHATWG, the ‘what wig’, they’re a standards body similar to W3C, to include observables as part of the DOM. The post has not been made yet. But the post is going to be made soon as long as everybody’s okay with it. And what it boils down to is the idea of using observables as part of event targets. An event target is the API we’re all familiar with for ‘add event listener’, ‘remove event listener’. So, pretty much anywhere you’d see those methods, there might also someday be an on method that would return an observable of events. So, it’s really, really interesting thing because it would bring at least the primitives of reactive programming to the browser. And at the very least it would provide maybe a nicer API for people to subscribe to events coming from different DOM elements. Because ‘add event listener’ and ‘remove event listener’ are a little unergonomic at times, right?

CHARLES: Yeah. They’re the worst.

BEN: Yeah.

CHARLES: That’s a very polite way of putting it.

BEN: [Chuckles] So, that’s one thing that’s coming down the pipe. Other things, RxJS 6 is in the works. We recently tied off 5.5 in a stable branch. And master is now our alpha that we’re working on. So, there’s going to be a lot of refactoring and changes there, trying to make the library smaller and smaller. And trying to eliminate some of the footprints that maybe people had in previous versions. So, moving things around so people aren’t importing stuff that were meant to be implementation details, reducing the size of the library, trying to eliminate some bloat, that sort of thing. I’m pretty excited about that. But that’s going to be in alpha ongoing for a while. And then hopefully we’ll be able to move into beta mid first quarter next year. And then when that’ll be out of beta, who knows? It all depends on how well people like the beta and the alpha, right?

CHARLES: Alright. Well, so if folks do want to follow up with y’all either in regards to the course or to upcoming releases or any of the other great stuff that’s coming along, how would they get in touch with y’all?

TRACY: You can find me on Twitter @ladyleet. But Ben is @BenLesh. RX Workshop is RXWorkshop.com. I think in January we’re going to be doing state of JavaScript under This Dot Media again. So, that’s where all the core contributors of different frameworks and libraries come together. So, we’ll definitely be giving a state of RxJS at that time. And next year also Contributor Days will be happening. So, if you go to ContributorDays.com you can see the previous RxJS Contributor Days and figure out how to get involved. So, we’re always open and happy and willing to teach everybody. And again, if you want to get involved it doesn’t matter whether you have little experience or lots of experience. We are always willing to show you how you can play.

BEN: Yeah. You can always find us on Twitter. And don’t forget that if you don’t find Tracy or I on Twitter, you can always message Jay Phelps on Twitter. That’s important. @_JayPhelps. Really.

TRACY: Yeah.

[Laughter]

BEN: You’ll find us.

CHARLES: [Chuckles] Look for Jay in the show notes.

[Laughter]

CHARLES: Alright. Well, thank you so much for all the stuff that y’all do, code and otherwise. And thank you so much Ben, thank you so much Tracy, for coming on the show.

BEN: Thank you.

CHARLES: Bye Elrick and bye everybody. If you want to reach out to us, you can always get in touch with us at @TheFrontside or send us an email at [email protected]. Alright everybody, we’ll see you next week.