The Frontside Podcast

Frontside alum and original podcast host, Brandon Hays, makes a special guest appearance to talk with Charles about the evolution of The Frontside as a company: where it's been, where it's going, and more hopes, dreams, and goals for the future!

Transcript

CHARLES: Hello everybody. Welcome to The Frontside Podcast Episode 100. Here we are. Episode 100. My name is Charles Lowell. I’m a developer here at The Frontside and I think it’s safe to say, your official podcast host. With me to celebrate the 100th episode, he was also here a few episodes ago but also was here on our first episode I believe, is the [inaudible] Hays. Hello Brandon.

BRANDON: Hi.

CHARLES: Welcome back to the podcast.

BRANDON: Actually, are you going to light your trainee badge on fire now in a bucket, in a ceremonial pyre?

CHARLES: I live in New Mexico, so I think I’m going to just after this, grab my shotgun and give myself a 21 gun salute. Just in my front yard.

BRANDON: There goes old man Lowell again, with the shotgun.

CHARLES: I’m just going to [gun shot sounds] in my own honor.

BRANDON: I was at the Alamo this weekend, actually. And I don’t know if it was just because it was fiesta in San Antonio but they had a demonstration, like a musket firing demonstration where those things are basically little cannons. They’re just small cannons. It’s very interesting. They’re very loud.

CHARLES: Yeah. They’re small, handheld cannons, yeah. So wait, were you – what is fiesta? Now, as someone who grew up in Central [inaudible], I feel like I ought to know this.

BRANDON: I don’t know. We found out by accident because we were planning a weekend to go hang out and get drunk on the riverwalk and we took our families down with some friends and then they’re like, “Oh, it’s fiesta,” which is like a 10-day celebration of the history and establishment of San Antonio – which I did not know is a 300-year-old institution. So, it’s like one of the oldest things in this entire western United States. So, it’s pretty neat. It’s different. It’s weird. It’s like 90 minutes from Austin. There’s nothing in Austin that’s older than six months. Every six months we must demolish something and then build a condo skyscraper in its place.

So, it’s kind of neat to be in a city where it has – walking around the Alamo, I’m realizing, “Wow. Setting aside any of the historical significance of Texas independence or whatever, this is just like a really interesting very old building. This is hundreds of years old in an area where there’s nothing that’s hundreds of years old.” So yeah, it was pretty cool. It was a good weekend and we got to see muskets being fired. And we saw a doctor gross my kids out by talking about the medicine of the day, in full costume and showing all of the procedures and threatening my kids with amputation. And it was a good time. We all had a good time. My nine-year-old thought it was the coolest damn thing he’d ever seen.

CHARLES: Really? Did the have bloody saws and everything?

BRANDON: Oh, yeah.

CHARLES: Was it like a reenactment of 300-year-old surgery?

BRANDON: It wasn’t a full reenactment. But it was a graphic description using the tools of the time.

CHARLES: Wow.

BRANDON: Highly recommend, check out the Alamo. Super fun.

CHARLES: That does sound really cool.

BRANDON: I did not expect to have a good time and it was a good time.

CHARLES: Yeah. Yeah, I know the whole reenactment with the musket firing is fun. And it is, it’s actually an incredible building. Although there’s been a big kerfuffle about something about how they’re going to preserve the lawn. But I haven’t really followed that too much.

BRANDON: Yeah. Yeah, I don’t care about the lawn. I care about – no offense, lawn, if lawn is listening. This is not weird, how Stanley broke our brains with the word ‘lawn’.

CHARLES: That’s true.

BRANDON: Yeah. He broke us real good.

CHARLES: Yeah. I can’t see a lawn without a beard.

BRANDON: So yeah. So, life has been pretty good, man. Let’s see. I left Frontside September, October.

CHARLES: 2016.

BRANDON: 2016.

CHARLES: So, it’s been months.

BRANDON: 18. Yeah, thereabouts, right? So, I assume that nothing happened since then and if I came back to The Frontside now, everything would be exactly as I left it. My posters are still up in my room. My Bon Jovi poster. You left my bed just as I made it, like kind of unmade. Everything is just preserved as a shrine to me.

CHARLES: Pretty much. I mean, we did give away the mics to Goodwill.

BRANDON: No.

CHARLES: We actually did not give away those mics.

BRANDON: I never even got to use them.

CHARLES: I know. Well, you know part of the problem is we don’t even get to use them that much either. It looks really cool and it plays really well, like our podcast studio. But you know, I’m now spending 75% of my time in Corrales, New Mexico. And at any given time, people are either working from home, or working remotely. So, a lot of times the podcast room tragically does not get used. But it looks so cool. People come in there and they’re like, “Wow, you guys must be really smart and technical people.”

BRANDON: I realize this is probably a rote stereotype at this point, but I am assuming the only reason that you moved is that you are dabbling in the production of meth.

CHARLES: Pretty much.

BRANDON: It’s like, I want to learn a new trade. Programming, it’s just – programming, how interesting does it stay honestly for 25 years?

CHARLES: Right. Yeah, and you know, we’ve got some good techniques. Continuous integration, deployment, things like that. Test-first. These are things that can be applied to different verticals. And I was looking…

BRANDON: [Laughs] We ship meth to production on the first day.

CHARLES: Right. [Laughter] Exactly. So, I figured it was a market ripe for disruption.

BRANDON: [Laughs] It’s probably true. So yeah, I wanted to ask you about that. You all kind of scattered to the four winds in some ways. You have Elrich in Boston and you’re in New Mexico most of the time.

CHARLES: Joe is in [inaudible].

BRANDON: Oh yeah, Joe moved to New York.

CHARLES: Yup. And honestly, the traffic is so bad in Austin that I’d say 50% of the time, people stay home rather than drive into our centrally-located office. So, that’s actually something that we’re struggling with right now because the bulk of the team is still in Austin. But the office space is underutilized. Our team size now, we have eight engineers. And five of them are in Austin. Our other staff is also in Austin. So, what do we do with the office? It’s a big question.

BRANDON: And that’s quite a cultural change, too. Because when I was there, we would tell people, “We want to be able to do remote someday. But we just don’t know how to get into that culture to change the way that we do our meetings and change the way that we do standups and coordination and communication.” I didn’t feel like we had the tooling at the time. So, something – I knew that at some point there would be probably a forcing function to basically catalyze something to allow that to work. And I’m curious to know what that process was like there.

CHARLES: I wish I could say that there was a process other than experiencing the force of the forcing function and then being forced into it and then just kind of dealing with it. I have not taken a poll of the other remote employees of which now I am one, at least for the time being. So, I don’t want to speak for them. But it was less painful than you might imagine. And the reason is because – and it’s one of those things you actually gave me this analogy back, probably three or four years ago and I love it – is sometimes you’re hanging off of a precipice and you don’t realize that you’re toes are two inches off the ground. And then all you can perceive is the precipice and you feel the weight of your own body concentrated on your fingers gripped to the ledge. And you don’t focus on the fact that you’re actually, the fall is only two inches long. And that’s kind of what we experience with the remote culture.

Now, I don’t want to say we were Pollyanna about it and didn’t realize that this was the step that we were taking and making sure to check in with the remote employees. But one of the things is our communication styles were already very asynchronous both for our client work, for our internal work, using mostly Slack and GitHub pull requests and issues – certainly for the development portion, very little changed. What we didn’t realize is that because of our involvement in open source, we were already acclimated to a distributed work style. We just didn’t really realize it. We didn’t have to change much. I think where we have a lot more work to do is kind of integrating people socially and making sure that conversations don’t happen that aren’t available for other people to consume asynchronously. So, if you’re having some architecture problem and you’re sitting next to somebody, you’ll take that avenue rather than let it play out in chat or over email. And there is definitely a certain portion of that, but I think we still do a lot of pair programming. That’s still our major mode. I’d say 75% of our code gets written as people collaborating. And so, while those in-office discussions do happen, the ramifications circulate rather quickly. And most of those are in the context of people pairing inside the office. Does that make sense?

BRANDON: Mmhmm.

CHARLES: So, I don’t think the office and the physical space were as much of a bottleneck as we thought they might be. And so, because of the – a lot of people did work from home already because of the traffic. And we were involved in open source. And our communication with our clients is usually – we don’t currently have any clients in Austin. So, that’s all to say that the transition was actually quite natural. And I think there’s some strong analogies between collaborating in open source and having a remote culture in your office. I think what we need to get better about is making sure that we get the team together at least twice a year, everybody together. Making sure that people are able to understand their priorities and get to circulate around and get introduced to a bunch of different people. And yeah, I don’t know. There’s definitely a lot of work to be done on the non-development front.

BRANDON: It’s interesting. The agile approach to things is to try something. I’m starting to think the agile and the scientific method are related where it’s like, “Here’s a hypothesis. Here’s the experiment. Here’s what we think we want to learn,” and then you learn it and you take the next step based on that information. And that failure is an option. I think that’s the point of agile, is to make failure safe because it’s small and you’re guaranteed to learn from it. Like, the point is to learn. And so, I really, I’m starting to think that those are just basically the same thing. That agile is like the application of the scientific method to product development. And it sounds like you’re being agile or experimental about your work. And the trick is, like any scientific discovery, the trick is in coming back around to it and analyzing it and deciding whether this was successful or a failure based on feedback and finding what the measurement was that you were trying to improve. So, the lesson there was, “Oh, people become disconnected from each other. We need to gather everybody for an all-hands periodically.” We didn’t use to have to do that because all-hands was every week, at least.

CHARLES: Right. Yeah, everybody was constantly – there was a constant chatter and you could just kind of, the context was just all sitting at that one table, in that one room on 38th Street. And all you needed to do was dip your ear into that pool of context and you’re set. Whereas that’s just not an option right now. So yeah. I think the danger with agile is not being concentrated in your experimentation. I think what gave us our fear about saying we’re going to do remote work – because I remember we always talked about it. We danced around the issue – was are we going to lose who we are? We have a set of way that we do things. And there is power in kind of sticking to the framework of the way that you do things. Because you understand it and you know it.

So, when you’re pushing and you’re experimenting, being able to say, “We’re going to – push and we’re going to focus on this one area and we’re going to iterate on it and we’re going to keep everything else static,” it’s going to be the wall that we can walk along. But we are going to push in this area. And so, I think the dangers of you doing that in all the areas of your business or all the areas of your project, you’re iterating and refining, nothing ever gets done. And so, it’s kind of like once you get to some ground that’s solid, when you do start iterating it, you start introducing instability. So, when you go remote you have to start thinking about remote work, whereas we didn’t have to think about that before. We were essentially, the feature of saying that we were a one-office company and an on-site company is we didn’t have to think about that problem.

BRANDON: One thing that you were just taking about is this idea of concentrating so that your experiments are happening one or two or maybe three at a time instead of trying to run five experiments at a time. And yeah, there’s another danger I think in agile of seeking local optimization where you’re basically like – it’s like taking a bacteria and running it through many, many, many iterations that’s targeting one thing and it mutates into this weird thing that only does this one thing. Or a dog breed that the whole – did you see that, I don’t know where this came from but there was some scientific findings that there was a dog that was bred in ancient prehistoric times that was bred to turn a spit to roast meat over. So, they bread a dog that the whole point of this dog was to turn a spit so that people could roast meat and go to sleep and let their dog serve it, cook for them I guess?

CHARLES: Wow.

BRANDON: That’s pretty impressive.

CHARLES: I would say like their dystopia is in the past. Or certainly canine dystopias. I guess we live in a canine dystopia.

BRANDON: Not in my house.

CHARLES: Not at your house.

BRANDON: This place is known as a canine paradise. So yeah, I think that’s a really interesting point though, that limiting the number of concurrent experiments so that you can actually respond to them in a meaningful way instead of just being like, “Wow, we learned a bunch of stuff we’re doing wrong. Anyway, back to the grind.”

CHARLES: [Laughs] Yeah.

BRANDON: Back to sucking at everything.

CHARLES: Right, right.

BRANDON: That kind of feeds into a lesson that I have learned very, very, very recently in the interview process for looking for my first real job in over a decade. And that process is very humbling. And one of the humbling experiences was being rejected for a job from a very notable larger former startup here in Austin. And their interview process is really buttoned up. I got really deep into the interview process and at the end of it they’re like, “Oh, you’re not technical enough.” And it was really, it was like, I don’t know. It was hard for me to process at the time but it’s super easy now to look back and go, “Oh, I was definitely not a fit for that type of job if being able to write JavaScript on a whiteboard without the aid of Google to solve problems and refactor code is like a fundamental part of what is valued in a manager there.” That’s just not going to be me.

But one thing I – and it wasn’t a colossal waste of time. There was a ton of time and energy I invested into that specific process, but I actually derived a ton of value out of it. Because every person I met there was focused on the same thing: their culture of making experimentation inexpensive so that everything there is framed in terms of an experiment. What’s the experiment here? What’s the hypothesis? What’s the expected outcome? How soon can we get to a place where we can validate that outcome? So, it’s kind of like everything is really lean. And yes, it does – like I asked, “What’s the dark side of that?” and it can lead to optimizing for a local maximum. So, you have to pause every once in a while and reflect at a larger scale. But it changed my attitude about a lot of stuff. I tend to walk around fearing failure. That’s more my speed. I’m afraid of failing because failure can be catastrophic. But that’s because I take big swings at stuff. When I go give a conference talk, it better be the best conference talk of my life. When somebody’s like, “Oh, that was the best conference talk I have ever seen,” I’m like, “Ah. I’m so glad you said that because if you’d said literally anything else I would have collapsed internally.” You know? The stakes are so high for everything. And making it safe for yourself to fail by treating things like an experiment and working with my teammates.

And so, two or three scenarios over the phone in a week when I was managing the team at my last company, somebody would bring something to me and I’m like, I instantly went to all the reasons this probably won’t work. “Here’s the problem with this.” And I thought, and I immediately turn around and went, “Wait a minute. Bring me a hypothesis and the experiment and how we can experiment with this thing.” And he’s like, “Well, we could try this next week and we’ll know whether or not this is a good idea.” And we tried it the next week. It was like organizing an architecture team because we were waiting to hire an architect. And the results were mixed for reasons I won’t get too deep into. But the fact was, it gave us the freedom to try things. And I’m trying to carry that spirit around with me now. It’s been really eye-opening. So, completely like, just a 4% alteration in the way that I think about problems, but it has the ability to dramatically alter the trajectory of how I solve things in the future.

CHARLES: So, do you include now inside the planning process experiments? Like, a certain number.

BRANDON: Absolutely.

CHARLES: So, the typical “enterprise” development is we have our features, we’re going to do them in this order because they’re this priority. And then agile comes along and it’s like, “You need to take these things and you need to break them up into small chunks so that they can be accomplished in small time slices,” so that you don’t basically bark up wrong trees. Or explore [inaudible].

BRANDON: Yeah, but that’s almost like a stupider version of waterfall.

CHARLES: But exactly. That’s exactly my point. Whereas the problem is, there’s no avenue for experimentation in there. Rather than saying the entire team is marching in this one direction that meanders around and focuses in on the local maximum, which hopefully is relative to the market landscape is the absolute maximum, saying, “We’re actually going to be marching in one major direction but we’re going to be sending out scouts at all points.” If you were actually – I’ve actually been reading a lot of ancient military history. And It’s just insane that an army, or even a detachment, would go all in one clump. They’re constantly sending out people. Information is really, really, really important.

BRANDON: That’s an extremely, extremely good point. I’ve actually – it’s so funny, because I’ve used a very similar description where we are trying to chart a course to this ocean of opportunity somewhere. And we can’t just send the whole team in a direction hoping that the ocean is in that direction. We have to have our Lewis and Clark. Somebody has to be the cartographer. Somebody has to be the explorer. And that means that there has to be a little bit more freedom for those explorers. I don’t yet know how to translate that into software terms. I just know that that’s a collaboration usually at most companies between product and development. That product is doing some of the exploring of the space and then development is doing some of the exploring of the technical capabilities and possibilities there.

CHARLES: So, you see it. What’s interesting is you see it in product planning, kind of in the large, with the waterfall. You see it in huge organizations. They have a research and development department. And I wonder if agile kind of saw the Balkanization of your feature set into very small component parts. Can you take the exact same principle and Balkanize your research and development and integrate it into micro-iterations? We have this R&D but we’re going to integrate it into our day-to-day and week-to-week process.

BRANDON: I think that is a really noble goal and I think I see some people making progress toward that. The company I interviewed with does it almost to a pathological degree where there is a point of diminishing returns where you’re sort of bound to this process of experimentation. And at a certain point you can only achieve incremental results.

CHARLES: Some of these problems, you just need to be able to think about them for a long, long time. I actually didn’t read, I actually didn’t see the talk. But everything from the title, Rich Hickey’s ‘Hammock Driven Development’, just that title resonated with me so much. I was like, “Yes,” because sometimes you just need to be in the hammock for six hours at a time. Or in the shower. Or hiking. Or doing whatever it is that you need to do to put yourself in a zen state where you’re just, your brain is slowly turning its wheels. And it can follow every lead to its conclusion without any interruption. And sometimes that process can take hours. Sometimes it needs to take weeks.

BRANDON: Right. I want to kind of pivot on that. Because that’s actually one of the biggest things that I’ve learned in the intervening time since leaving Frontside, which is creating space instead of trying to maximize – one thing that I did when I was at Frontside and then did again at my next place and I’m realizing is really has long-term negative implications is cram as much into a work day, as much output out as possible. I’m very output-oriented. I want to jam as much into my day as possible. I want to jam as much software out the door as possible. And people describe working at Frontside while I was there as one of the most intense work experiences they'd ever had. Literally, I can project that, literally just from my own intensity of trying to cram all that stuff.

And providing that space for developers to ruminate on hard problems, on some of the harder problems they encounter, providing space for managers, I’ve learned that a big chunk of what it is to be a manager is to be available. And so, I actually want to write a sign – I was on the fence about doing this but I think I’m actually doing to do this – I have an office and I’m going to write a sign and put it up on the door that says, “If I look busy, interrupt me and remind me I’m not doing this right.” So, creating the space to ruminate or to be available for discussion, people that protect their breathing room sometimes are made fun of, especially in American corporate culture. I walked in and they were just reading a newspaper. What the heck are you doing at work if you’re just going to read a newspaper? Like no, this is actually really important time.

CHARLES: I think it’s, yeah, it’s something that I think about a lot. And I know I’ve shared this analogy with you before. I don’t know if I’ve done it on the podcast. But I saw and I can’t take credit for it. I actually saw it at DevOpsDays I think in 2013. There was a woman giving a talk and she was just talking about managing developers. But one of the things that she was saying was that if you looked at a microservices architecture or you looked at just even your operating system, and if your CPU was constantly pegged, you were squeezing out 100% of every time slice, instructions were just flowing through that, you’re going to have a very unhealthy, very brittle, very prone to failure software system. If our microservices were not available to actually service requests, and service excess requests, and service spikes of requests, then something is fundamentally wrong.

BRANDON: I want to add to that a little bit, because the thing that I noticed in managing a team where I received a ton of pressure to peg everybody out at 100% – and it jived with my philosophy at the time of, “Hey, I’m 100% guy. Everybody I work with is 100% type people. And then, let’s peg everybody at 100%. This is a startup. Let’s get everything going,” and I realized very, very quickly that if you don’t preserve a little buffer, 20% buffer in that level of intensity, there is no ability to share resources. Everything is now a silo. So, if you’re going to peg all your CPUs out, part of that thrashing is that there’s no time for people to share things with each other. And people become very protective over their little silo all of a sudden. And it causes us – it’s actually like the first stage of a catastrophic cultural collapse if everybody’s pegged out at 100%. And literally, just dialing down the intensity is often the only thing that’s necessary to get people to feel comfortable sharing some of their time with each other. You do a really good job of that with the lunch and learns. You mentioned that y’all are doing better thoughtful lunch and learns and stuff like that. It’s one of those forcing ways that you can force that and say, “Hold on. Stop the development and do some stuff where you’re actually sharing things with your teammates.”

CHARLES: Yeah. And we do that. My biggest concern is that that actually increases the intensity. So, one of the things we’ve done is we used to actually be very formal about our lunch and learns. It’s like, “We’ve got to generate content and put it out on the web so that people can see us.” We backed away from saying – we’re not going to do them as often and make sure that people can actually do them. Yeah, making sure that people don’t feel overwhelmed by, “I’ve got a lunch and learn coming up.” The point is to share something that you’re passionate about and maybe introduce some really cool ideas to ferment in people’s head. Rather, that’s kind of the goal. There are certain things that we do very much feel interested in generating content. But I think, we’ve kind of been dancing around the ideas of distributed computing and IoT and what are some of the others?

BRANDON: If you say blockchain, I’m going to just virtually punch you in the face.

CHARLES: [Laughs] I actually didn’t. Did I say blockchain?

BRANDON: No. I just was waiting for you to say it.

CHARLES: Okay, no. I haven’t. Well, because that’s – but it is distributed computing in Web 3.0, right? These problems – and we’re actually going to be podcasting about this next, so in two weeks you can tune in to listen to us talk about blockchain but in the context of distributed computing – and one of the things that we’re seeing is now we’re starting to pay the price of outsourcing all of our lives to these central services like Facebook and Google and Amazon. And I think now they’re starting to build a credible and more mainstream movement to wrestle back that control and say, “What would it mean to have software as a service that wasn’t actually dependent on some central thing?” What would it look like to have Slack where it’s Slack that looked like email? Where everybody had their own email server, maybe not a bad example. But you’ve got an email at Gmail or Microsoft or Yahoo or your company-run that’s big enough its running its own Outlook client or something like that. Email is actually a really great example.

Now probably people are going to crucify me for saying this, but I think it’s actually a good example of a distributed system that’s worked well. I own all of my email. All the messages that you send to me, I own, and all the messages I send to you, I also own. But you also own the messages that I send to you. Information is duplicated. And it’s fine. If I send you an image, yes it’s on your hard drive or it’s on your Google Drive. You send a message to me, it’s got an attachment, I also have that attachment. But the point is that we can each own our email and we each own our email service. And we can change it up. That’s not possible with Slack. That’s not possible with Facebook. That’s not possible with all these other sharing platforms. All of them are controlled by this one thing. And so, I think that that’s something that we've been exposed to through the lunch and learns and I’m actually certainly very excited about it. It’s not something that we’re going to be investing in immediately. We’re kind of dancing around that idea. But that’s something that’s come out of that. So yeah, we’ve kind of refocused it on, what is something that you feel good about?

But back to the original point, I think that this is something that applies on all fronts. If you have a business where you can’t actually take opportunities because you don’t actually have people – so there’s maxing out at the individual level, filling up people’s workspace with client work or filling it up with what have you or having them work nights and weekends. There’s individual maxing out but then there’s like maxing out of your business. So, if you have – we’re a consultancy – if you have 100% utilization or you’re shooting for 100% utilization, that everybody is placed on a project, that is a brittle and unsustainable system.

BRANDON: I wish you would have told me that 18 months before I left there. There were like two years where we were at 100% for two solid years.

CHARLES: Yeah, yeah. We’re still at 100%.

BRANDON: Yeah. I wonder what would have happened if we’d had a little, if we had figured out how to build in space.

CHARLES: Part of the problem – so, here’s the thing though. Space, nice space costs nice money.

BRANDON: Yeah.

CHARLES: And so, that’s the thing, is you have to charge more. And you have to say, “We are going to be more expensive than other people.” You have to be dedicated to be at the forefront of a cultural battle, essentially. In the same way that people were with testing, where it was very [controversial].

BRANDON: Yeah. You were with CI. CI is a given now, right? CI is…

CHARLES: Yeah, like [inaudible].

BRANDON: This idea was semi-revolutionary when you and I were talking about this in 2012, 2013, that we ship to production on the first day. We don’t even start building software until the CI system is set up. The first thing we do is set up Jenkins and tests and get everything, the pipeline working. And now, that’s just what people do. By and large, that’s how software is expected to be built. And the tooling has really come up around that. But that was an expensive way to sell software five years ago, that, “Hey, this is going to cost more than bringing in Cowboy Bob and having them come jam in your console for 40 days and ship a bunch of stuff that then will most likely collapse and you won’t know about it and Cowboy Bob has ridden off into Juarez, Mexico.”

CHARLES: Right, with his saddlebag stuffed with your cash.

BRANDON: Yup.

CHARLES: Yeah, no. So, you have to – the problem is, you know when you pick these battles, you need to be prepared to fight the war of attrition of they’re not going to be able to perceive the value for six months, a year, right? You’re going to have to ask your clients to bet on this strategy. And it’s a bet. And you’re going to have to say, “It’s going to pay off in six months. It’s going to pay off in a year.” And you’re really going to start raking in like five years. That’s when…

BRANDON: Yeah. Try making that pitch to a startup founder that is borderline, that is on the verge of an anxiety attack, and you can kind of just figure out what my last year was like. And the…

CHARLES: So, that’s one of the reasons we don’t really work with startups anymore. They have a five-year plan, but not really.

BRANDON: Yeah.

CHARLES: They’re fighting for their survival. And they’re fighting for the opportunity to have a legitimate five-year plan. And so, in that sense, it’s maybe not a good fit for the way that we develop software, because you either need an extraordinarily prescient founder who has been through this before, knows the true costs of software development, and is pretty well-funded so that they can actually – because we’re more expensive upfront, like a lot more expensive upfront and so sometimes they flat out don’t even have the cash. And that’s something that you can make a quick, “It’s not a good fit,” but then there also needs to be this understanding and an acknowledgment that what you’re really shooting for is your five-year dividend.

BRANDON: Yeah. It is really interesting, the turn that occurs when a company finds product/market fit. By then it’s too late to fix the problems. So, it’s really tricky to find the balance of: how much energy do you put into the success case for a company before they have product/market fit? How much time and energy do you invest in betting that this is going to be successful versus betting that if it is successful, hopefully we’ll have the time, money, and resources to redo a bunch of the things that we are going to have to apologize for later? And I think that’s what makes…

CHARLES: Right. Like, where do those two lines cross on that graph?

BRANDON: Yeah. Because you and I have both seen startups completely sunk by somebody who was overly focused on building a scaleable architecture in a company pre-product/market fit. That is a common story where an engineer that doesn’t understand the business value of what they're doing and only focused on “quality” will absolutely torpedo, they’ll chew up your first million and a half of funding and leave the place in just a smoldering pile of ashes at the end. So, it is tricky. It’s totally a difficult thing.

But I think coming back to your point of being sort of a vanguard of cultural, the tip of the spear on somebody’s cultural changes – DevOps would be one. People that were really investing in DevOps culture in 2010, 2012 saying, “Hey, this, automation, is the future of how software gets shipped, maintained, observed, supported.” And so, now it sounds like, so what is your big bet for the future?

CHARLES: Boy. That’s a great question. There are two bets. One you’re going to like, one you’re going to vomit.

BRANDON: [Laughs]

CHARLES: But that’s okay.

BRANDON: Yeah. I don’t work for you.

CHARLES: You need to serve, what is it? You need to serve the spiny urchin with the yellow tail.

BRANDON: Is that a Sonic the Hedgehog reference?

CHARLES: It’s just a sushi reference.

BRANDON: Oh, okay.

CHARLES: Some people don’t like urchin. Or maybe they don’t like eggs. What it like, the roe that come with sushi. But they’re on the same plate.

So, I would say the first one that I’ve been thinking about a lot is optimizing for capacity and being able to handle spikes and not being at 100% both for people and for utilization. I think that’s something that is – I don’t see how you could have a healthy software development process if people are completely spiked on delivering, heads down delivering features for product. That is something that I’m betting on. Essentially, you could call it the 25% time but it’s really about having excess capacity to exploit opportunities as they arise. And then being protective of that excess capacity. Because you can exploit an opportunity. Your CPU has a spike load up to 100%. But then make sure you [inaudible] down to 50% at some point, or 75%. And so, I would very much like to see Frontside have a bench where people can rotate out and they’re working on different stuff that are not even client-related. They can recharge their creative tanks. They’re not going to be idle.

BRANDON: Yeah, I’ve really come around on – and I really hated this at the time – but I’ve actually come around to the thoughtbot style of working on a product where – because owning and managing a product and developing it as a side quest, the goal is not necessarily for that product to catch fire and become the world’s next big thing and to replace your consulting revenue. The goal is to give people a sense of – think about all the stuff that you’ve learned in your side projects that you went back and brought to your work. And some of my biggest gains as a developer have come from having a side gig of some kind, some side project that that’s how I learned Ember. That changed my life. And I would never have gotten to try it if I was waiting for somebody at work to tell me it was okay to do it. So, it’s about taking that permission back for yourself and giving yourself permission to try stuff.

So, it could be something like that, or it could be the content stuff that y’all do. Or it could be conference talks. It could be whatever. But the goal isn’t necessarily to produce things that have a direct return. It is to create the space to allow people to flex some muscles of creativity that you may not get in your day-to-day work. And that’s very difficult to offer to people in any company. Now having explored startups and larger companies, but I would say especially in a consultancy where the exchange rate is dollars for days. It’s sort of like when I was freelancing. I could feel every vacation I took draining both real money and opportunity money out of my bank account. That’s such a hard, difficult thing to do. And so, you actually have to create the budget ahead of time and say, “This budget is allocated to these things and it’s already spent.” Anyway, that’s really tough to do.

CHARLES: It is hard.

BRANDON: If you can exercise the discipline necessary to do that and create the environment for that, I would say you’re ahead of 90% of companies in the industry.

CHARLES: Yeah. Yeah, so that’s something I definitely want to bet on, because I think that’s where the best things come from.

BRANDON: Okay. So, what’s the thing I’m going to hate?

CHARLES: Functional programming.

BRANDON: Oh, Charles. Okay, I have to stop you. Do you know what I’m doing? Did I tell you this yet? That I am participating. When I told them this, I was like, “Charles is going to have a field day with this,” but I am participating in a Haskell study group.

CHARLES: No way.

BRANDON: And I’m like four exercises into this thing. I have to do four more for next week. And I’m like, “This is bizarrely easy, actually,” after as much JavaScript as you and I did in sort of a functional style and then learning Elixir. And I was like, “Wait a minute. The case statement is, Elixir just stole Haskell’s case statements.” So like, so far I’m not finding functional programming to be onerous. Or anyway, but we’ll see when we get to the static typing. But so far, I’m not getting any of that in the earlier lessons of the book.

CHARLES: Yeah, the static typing. But the thing is, you can do – it’s not 100% necessary. It isn’t in Haskell, for sure. But I’m surprised. What inspired you?

BRANDON: We have an architect at the office that was like, “Hey, I want to do sort of a functional programming book club.” So, we have a Slack group for FP study group.

CHARLES: Are you doing ‘Haskell: From First Principles’?

BRANDON: No. That one was a little actually intimidating.

CHARLES: Really?

BRANDON: Yeah. It gets into the lingo a little early. And we’re doing one called ‘Get Programming with Haskell’ that is a little more – ‘Haskell: From First Principles’ is kind of math-oriented. So, for somebody with a math background but not necessarily a programming background, it’s perfect. But for somebody with a programming background that is just trying to understand functional programming principles using Haskell, ‘Get Programming with Haskell’ is actually a really great option.

CHARLES: Okay. Actually, I have not heard of that one.

BRANDON: The stuff that I’m looking at looks just like Elixir. So, it’s early. But it’s very comfortable so far.

CHARLES: Yeah. So, this is the thing. It’s all a matter of messaging and marketing. Because I really feel – so, it is like there are a lot of behaviors that you see sometimes in currently entrenched functional programming communities that I think are, well I think they’re objectively repulsive. But I think they’re also pragmatically repulsive and that they repulse potential community members. But I think a lot of it too is people talk about these things that are, they use abstruse terminology. And they’re kind of chattering back and it’s very jargon-oriented. And there’s just – people operate with a different set of concrete things. So, when you and I are talking, for example we might talk about a Rails controller and that’s a very concrete thing. You know exactly what I’m talking about. It’s something that you have held in your hand, literally. Remember when we got that Rails codebase that came as a thumb drive?

BRANDON: Yes I do.

CHARLES: But the point is you knew that this had a Rails codebase on it. There were any number of controllers. And when I say controller to you, a controller is an abstraction, but not really. Once you work with an abstraction long enough, it becomes concrete. And so, part of the problem is just a mismatch in language where people are talking in their world about concrete things, things that you can touch and you can feel and you can exchange and they’re very relatable. But from another person’s perspective, they’re talking about something that’s totally abstract and totally opaque and totally what have you. And so, I feel like yeah there’s a huge mismatch there. And that’s been one of the big bets.

The other big bet that I’m making is on this trying to make what is currently abstract to JavaScript and Ruby developers be concrete. And I think that we’re going to see type classes like functor and monoid and semigroup and all these things, they’re abstract to you now, become concrete over the next five years. And so, that’s something that I’m betting on.

BRANDON: Check out this – and I know that you have a good relationship with the people that did the other book, but it really does tend to come from more of a mathematical background. And this one actually does speak to people with JavaScript, Ruby, Python experience. Like, “Hey, here is how you will perceive these things.” And so, it’s much more approachable. I’m still in the first unit of the book. But having sort of tasted it a little bit, it’s like, “Wait a minute. This is actually extremely familiar and not super intimidating.”

CHARLES: Exactly. And that was kind of – so, I read the other book. And I think I was also aided by the fact that I tried to learn Haskell probably for five times in the past. And so, I also had the benefit of jumping against the wall with the velcro suit and bouncing off four times. And fifth time, it stuck. So, I had just temerity on my side and a general feeling. But that’s definitely – the lesson that I actually came away from reading that book was like, “Oh, there’s a mismatch in concrete concepts.” It’s using concrete concepts that are concrete to people with a CS background or mathematics background, or people who are brand new. Honestly, people who are brand new to programming who don’t actually have JavaScript or Elixir or Ruby or any other thing to lean on, I think that the First Principles book is actually pretty decent for them, too. Because they don’t have anything to compare to.

BRANDON: They don’t have anything to unlearn.

CHARLES: Yeah, they don’t have anything to unlearn whereas one of the things I took away was I was like, “Oh, man. I’m using semigroups all the time. This is something that I do constantly.” When I’m coding, I might do it eight times in a day. I just didn’t have a name for it.

BRANDON: Right. They’re like design patterns, just at a micro level.

CHARLES: Yes, micro-design patterns. Yeah, it’s like a RESTful architecture for your code. In REST you only get five verbs. There’s five methods, man. That’s all you got.

BRANDON: Okay, so those are two bets. And I want to cover one more thing because I know we’re super overtime. But the last thing I want to be able to say about talking about what we’ve learned since I left Frontside but I want to put a bow on that. So, the two things that you’re betting heavily on are functional programming as a basis for solid architectures in the future, like the work that you all are doing. And…

CHARLES: I would also like to say, and this is something – let me just add one more thought. What I don’t understand, and this is in no way like, I don’t understand people who do the, “Saying goodbye to framework X.” That’s not me with object-oriented programming.

BRANDON: Often abstractions are like oversimplifications but they’re really useful, sort of like Rich Hickey’s Simple versus Easy. Like, “Hey, there’s a lot of promise with that metaphor. It’s a leaky abstraction but it’s a useful abstraction.” And Gary Bernhardt’s ‘Functional Core, Imperative Shell’ is a leaky abstraction but it’s a useful abstraction. If people haven’t seen or experienced that, it’s pretty good. The subtlety is that these are tools that are suited to certain situations a little better. And those same situations can exist in the same codebase, can exist in the same program.

CHARLES: Yeah. I still, I love Ruby. I adore it. And in some ways, I’ve been researching functional programming and it’s been going on for the last four years. So many times, people are like, “Oh, I just can’t stand this tool anymore.” And I’m like, “Man, I still love Java.” I don’t understand how learning to love something decreases your love for something else.

BRANDON: That happens the first two times that you fall in love, is that you feel like you have the old thing less in order to love the new thing. And then you start realizing, “No, you are allowed to fall in love with new things without falling out of love with the old things.” I would almost use that as an interview question. Is there some way to use that as a way to gauge somebody’s actual real concrete maturity as a developer? Because that is a mark of maturity.

CHARLES: Yeah. I mean, you could say, “What’s some tool that you no longer use that still informs your day-to-day routine?”

BRANDON: Yeah. I guarantee you, people that were doing Smalltalk in the 80s think about it all the time.

CHARLES: [Laughs] Yup. Yeah, exactly. Exactly.

BRANDON: Alright. So, I want to cover one last thing.

CHARLES: It’s part of growing, right? If you’re going to grow as a developer, you can’t be shrinking at the same time as you’re growing. Otherwise, you’re like the same size, just in a different place.

BRANDON: However, you don’t get any Medium think piece points. Nobody does the one clap, two clap, forty, for blog posts that are like, “Why I’m still using some programming language but using one a little more than I used to use and this one a little less.”

CHARLES: [Laughs] Zero claps.

BRANDON: Yeah, zero claps on that think piece. I just want to cover one last thing before we wrap this up, and it is the fact that Frontside, the biggest gift that Frontside gave me was the mission for the next 20 years of my career. I think it could change, but I’m pretty confident about this, at this point. Being approximately 20 years into my career, I feel like I kind of have a feeling for what the next 20 years is about. And the Frontside really drilled that into me and helped me focus it and helped me dial it in.

And it is this idea that there is an incoming generation of programmer that thinks about things differently than the previous generation in a pretty radical way. Because the previous generation all came out of the same schools. They all look the same. They all have a similar shared set of values in general. They created the Sil- – you know, I’m not actually going to be overly critical of the Silicon Valley culture that exists now. It is a result of the type of people that came out at the time that value innovation over almost anything else. People talk about ripe for disruption. The fact is, that has been an engine of economic growth and progress for society in a lot of ways that has a lot of costs that weren’t factored in by a bunch of people who all thought the same way.

And now, with people coming through code schools and people coming from different backgrounds and people coming from different environments, they’re looking at programming and software as either an economic opportunity or something they didn’t see that they could possibly do. Those doors were not open to that group of people before. There is a natural influx of people but many of them are bouncing out because they’re not finding that group of people, they don’t have a shared enough set of values that the people that are new are coming in and finding job opportunities, finding promotions, finding leadership positions.

And so, I know now that my mission over the next 20 years of my career is to create those opportunities for people that have different backgrounds from me and different experiences. The career tracks, the promotions, endorsing and supporting and kind of sponsoring this incoming group of freshmen into our industry that come from different places, different backgrounds, different problems that they care about solving. They want to figure out how to solve the Flint Michigan water crisis instead of delivering socks to people in Silicon Valley, you know? So, I feel like we’re at the beginning of a seed change in the value system potentially of our entire industry. But that’s going to require training up the next generation of technical leadership.

And I felt like the best thing I could do right now is learn to be a better manager, because I really like that job. And it provides the opportunity to find, hire, sponsor, promote and encourage those people to move into their own leadership positions. There are lots of other things that a person, you could be a VC and care about that stuff. You could have lots of different positions and put yourself in a position to do that. You could be a consultancy owner. You know what I mean? There are jobs that you can do that you can accomplish that goal. But it gives me such a sense of direction that when I’m looking for a job, I was looking for a home for that mission rather than just the thing that I felt like doing. Like okay, this job is important to me because I need it to house me and this mission so that I can support my family but have enough emotional overhead to participate in community stuff, but enough ability to lead within an organization, enough influence to actually push that agenda. So that the next generation of people are making better companies.

So anyway, all of that came out of my time at Frontside where you and I sat around talking about: how do we build a place that is like a monastery? These were your words. You remember this? We want a monastery for code where people can just focus on becoming better developers. And underneath that though was the sense that this was a place of opportunity for people that might go somewhere else and stagnate as a developer. This will be a place to accelerate them. And so, that kind of spun me out and accelerated me into my mission. So anyway, I just wanted to point out that that was like, with a bullet, is the most important thing that came out for me in my time at Frontside, was that it clarified for me what I was trying to accomplish with the next couple of decades of my career.

CHARLES: Wow. Well, that’s fantastic. You definitely did a lot of that both here at Frontside and I mean you’re continuing to do that. I definitely want to see more public speaking from you. Maybe some [inaudible] perfect. [Inaudible] at EmberConf was actually fantastic. But I mean, you’re also able to help people find their mission, too. Like the talks you have at Keep Ruby Weird and even really, the first talk you gave at LoneStarRuby about moving Ember. It’s always, how do I adapt what I’m feeling to my overall mission and then relate that back to technology? Man, I just can’t wait. I can’t wait. When are you going to hit the road again?

BRANDON: I think this is the year. I’m going to start thinking about this stuff. I’m looking at the stuff that I wanted to talk about on this podcast and I was like, “Oh no, wait. That’s like a dozen podcasts.” Like, no. Absolutely not. Not possible. I will say, I miss so much, this time that I spend with you. I don’t want to let it go. I really miss working with you. I really miss having these conversations whenever I want. This has been a very, very special privilege for me to be able to do this with you today. And congratulations on Frontside continuing to thrive and grow and become more of its own entity and more of its own special flavor. And it makes me really happy to see the people coming out of there, that it’s still doing its mission of making great software by making great developers. It makes me real happy.

CHARLES: Yeah, yeah. Hopefully we can keep on keeping on. I do miss working with you. I miss the conversations that we would have in the kitchen which are basically an extension of this podcast. But I also, man, I really, really, really, really like working with the group of people that are here today. I’ve just seen them producing just some absolutely amazing things. And honestly, there’s a selfish aspect to it, too. I get stimulated. My own thinking and learning is stimulated by the people that I work with. And like I said, the whole side note we had about distributed systems and IoT and just a constant ferment of things. So, I still really, really, really enjoy it.

BRANDON: That makes me happy.

CHARLES: And I’m really glad that we got to kick it today.

BRANDON: Yeah, me too.

CHARLES: I thought you were going to say that your 20-year mission was to have your perfect Emacs initialization setup.

BRANDON: Oh my gosh. Some of these days, I’m going to figure out RuboCop.

CHARLES: Actually, do you want to pair on that?

BRANDON: Yeah, let’s do that.

CHARLES: Alright, everybody. I’m going to sign off. If anyone wants to continue the conversation, obviously you can get in touch with Brandon. He is misspelled @tehviking on Twitter. T-E-H-V-I-K-I-N-G. Always come at him.

BRANDON: Don’t @ me.

CHARLES: [Laughs]

BRANDON: I work for a really cool company and if you ask me about it on Twitter, I’ll tell you all about it.

CHARLES: Awesome. And we of course are Frontside. You can get us on Twitter at @TheFrontside or just drop us a line to [email protected]. And we would love to talk to you more about this podcast and all the wonderful things that we do here, which includes building custom software that you can stake your future on, that’s going to be good for the five-year outlook. So with that, goodbye Brandon. Goodbye everybody. And we will see you…

BRANDON: Bye Charles. I love you.

CHARLES: Me too.

Taras Mankovski: tarasm

In this episode, Taras and Charles talk about a project that they work on together: Funcadelic - a Functional Programming and Category Theory for Everyday JavaScript Development.

Funcadelic takes the simple idea that a single function can operate on many different data structures (typeclass oriented programming) and brings it to JavaScript. Yes, there are a lot of FP libraries out there, but this one is geared towards unlocking the magical powers of functional programming while always maintaining a tangible and accessible experience for JavaScript developers. Because if you're a JavaScript developer today, you're already using most of the structures in funcadelic!

Transcript:

CHARLES: Hello everybody and welcome to The Frontside Podcast Episode 99. My name is Charles Lowell, developer here at The Frontside and your podcast host-in-training. And with me today is Mr. Taras Mankovski. Welcome.

TARAS: Thank you, Charles. It’s a pleasure to be here.

CHARLES: Yeah. So, you are ubiquitous in the JavaScript world. You do a lot of stuff with mentoring and you are involved in a bunch of different interesting projects. I think you’re one of those developers who’s difficult to classify, which is – that’s definitely one of my favorite kind of developers. I wanted to have you on the show today because there’s been a project that we’ve been collaborating on. And there have been some interesting ideas to come out of that and solidify through that project, at least in my head. And yeah, I thought we could maybe just talk about that a little bit.

TARAS: Yeah, sounds good. It’s going to be fun.

: The thing that we are going to be talking about is a project called Funcadelic. It’s more than really just a library, a JavaScript library on GitHub. It’s kind of a different way of thinking about your code. And so, I know for me, where this really became part of my workflow was, when was it? It was about three months ago or something like that? Six months ago?

TARAS: Oh, I think yeah, I think it’s probably more six months ago. I think it’s probably what, two months, I think probably December maybe?

CHARLES: Okay. But it’s hard now to imagine working without this tool on my workbench. It’s been probably the biggest game-changer for me in the last, I don’t know, definitely in the last several years.

TARAS: Yeah, it’s pretty impressive how little, how small of a library can have such a big impact in what we do day-to-day. Because it definitely makes me think differently about how I can solve problems that I solve on a daily basis when I work with React. So, it’s been pretty interesting. I think for me, having worked with this library, I think what I’m getting is an understanding of how things work in a way, and a perspective on how React works, in a way that I don’t think was available to [inaudible] Funcadelic. The funny thing is it’s not a React library, right? It’s not designed for React. It’s just that…

CHARLES: I don’t even think that – it helps you think about React, but I don’t even think it’s the way that the React developers think about React, right?

TARAS: Yeah, I don’t think so, either. I think a lot of people are on the spectrum of understanding functional programming. And I think a lot of people use, people learn how to use React, but they don’t really – I don’t think a lot of people have traveled very far. I’m talking about general, majority. There’s definitely people who know functional programming really well. And there’s a lot of really good libraries in the JavaScript space for doing functional programming in JavaScript. But I don’t think the general public, the general people that on a daily basis go into – write a render function and do ‘this.’ or like ‘product.map’ and then return an array of components. I don’t think those people necessarily think about or get the context within which they use this tool.

CHARLES: Right. And I think that’s actually kind of one of the reasons I think a library like Funcadelic is so important and fills kind of a missing piece in the ecosystem, is because it really is predicated on the idea that programmers use these concepts all the time. They really are, they’re foundational. But we only kind of see them out of the corner of our eye, in our peripheral vision, as being like a formal concept, like mapping. And giving a name to that. You know what I mean? Like you do the mapping, but you’re not thinking about: how do I generalize over it? And I think that that for me, certainly in my journey with functional programming, I thought that it was mostly about functions. Not to say that it isn’t, but that was kind of the end of the story. It’s like, keep your functions pure so that the outputs are only dependent on the inputs. And away you go. And understand closures and higher-order functions, functions that return functions or take functions, and that’s it. But I really feel that that’s only half the story.

TARAS: Part of it I think is that for people, even if you look at the kind of content that’s available around functional programming, it tends to be – trying to kind of [reach] people into this idea of thinking of map, filter, reduce, kind of operations. And I think that’s a place where everybody starts. But I think what happens is that you really are missing – and I think for most people. And it wasn’t for me, it wasn’t until you wrote the readme for Funcadelic and then I read it – up until that point I didn’t really, I was also the same. I didn’t know how these things were related to each other. Because there’s this history and wealth of conceptual depth that connects all these things together. And these things are well-understood by people who don’t – they’re probably not writing JavaScript on a daily basis. They might be like Haskell programmers or Lisp programmers or ClojureScript or something like it. In other worlds, not JavaScript world. So there is all this understanding about how functional programming works but I don’t think it’s really leaked to the general masses of the JavaScript community.

CHARLES: Yeah.

TARAS: You know? And it wasn’t until I started reading the readme – I’m like, “There’s so many answered questions that I didn’t even know these questions were asked.” You know?

CHARLES: Yeah, yeah. Yeah, no. And I think you’re absolutely right. It isn’t accessible to the general JavaScript community. And part of that is because one person’s – like when you read about these things, when I would go read about these kind of higher-order concepts, of basically classifying functions, not just of saying, “Yeah, what is the essence of a map operation? What’s the essence of an apply operation?” you know, “What’s the essence of concatenation?” things like that, I go read the documentation in Haskell or in Clojure. And first of all, it’s hard to distinguish when you’re not programming in those day-to-day, am I reading reference documentation or explanatory documentation? But even in the explanatory documentation, they’re using what seems like incredibly self-referential and abstract examples. And I don’t think that’s necessarily a knock against those communities.

I think what it is, is what’s concrete to one person is abstract to another. And it’s like, if you’re working with those things, you’re working with those sets of analogies, and then you’re working with those abstractions every day, then they’re concrete to you. In a sense that once it clicks in your mind and your mind kind of accepts it and rationalizes over it, then it moves from being, “Yes, it’s an abstraction. But it’s a concrete abstraction,” in the sense that you can have a conversation with somebody and use that abstraction as an example, as a counterpoint, and a method triangulate and reveal other abstractions. But if you’re talking to somebody for whom those abstractions haven’t clicked yet, then it’s just, it’s opaque. And it’s not helpful. And so, I think that one of the things that I realized is like we are using these abstractions, we just don’t have names for them. And so, I wanted to give them names and put them in the hands. And the names are weird, but they are really useful. And so yeah, maybe we could talk about some of those right now. Because I think that maybe now’s a good time to actually introduce some of those abstractions.

So for example, if you don’t know what a functor is, it’s worthless to talk about a monad, in my opinion. So, that was critical piece of information for me. Because that is like missing in every monad tutorial you ever read. At least, I must have read a thousand monad tutorials. And they kind of glossed over functor or didn't mention it. Whatever. Maybe they did but I wasn’t looking. And that needs to be put front and center, that there is a natural sequence to these things. It’s like, some of these abstractions are built on other abstractions and you have to – you can’t skip to monad. You have to start with functor. Again, I realize that’s probably gibberish gobbledy-goop to a lot of people. So really, this is what Funcadelic is about, what this conversation is about, is just saying – talking through it in real-world examples to make those abstractions concrete, so it doesn’t feel strange anymore.

TARAS: Yeah. It’s definitely giving names to things. I think it’s really helpful. One thing I really like about Funcadelic is that you’re not giving names that you made up. These are names that existed for 50 years. They’re historic. And so, when you talk to somebody who is familiar with functional programming and if you say ‘functor’, all of a sudden, I feel much smarter. And we’re actually referencing the same thing, because we are – because alternatively, you can say something like, “Something that is mappable,” right? Like a functor is essentially describing something that you can map over.

CHARLES: Right. That’s all it is.

TARAS: Right. But you know, having a name for it, it allows you to just describe it exactly as it is.

CHARLES: So yeah, there are tons of things that you can map over, right? Most of the time, we think about arrays as something that we can map over.

TARAS: One thing I found really interesting in starting to use Funcadelic is that when you start thinking about things as they are like abilities – like you know with an array you can map over an array. It’s something we’ve all been doing for a while – but then something that you end up doing a lot of. When you get familiar with an array, being able to use object map, mapping an object, becomes something you want to do at some point. Most of the time, what happens is that you’re like, “Oh, I don’t actually – well, I have to write this thing. How do I write this thing?” and then by the time you do this, you’re like, “I’m just going to go to Lodash and I’ll just get the map thing that will map an object.” At the end of the day, it doesn’t quite necessarily feel right because a lot of these libraries – like, Lodash has map but it feels like there is always some kind of a compromise with how these things are implemented. It’s not consistent.

CHARLES: Right.

TARAS: And I don’t remember exactly what the problem with Lodash map was. I know for a fact there are, like there’s different ways that you can map things. There are different functions available for mapping different things in Lodash.

CHARLES: There’s ‘map to’ and ‘map in’ and blah, blah, blah, blah, blah.

TARAS: Yeah. All those different variations. But I think it’s been really interesting. We’ve been using Funcadelic on a project we’ve been working on, on microstates. And just being able to use one function map that is going to map an array or is going to map an object and it’s going to do it the same, use the same function, and it’s going to do the same thing.

CHARLES: Yeah. You have one function that maps over the values. And that’s the thing, is you realize you can map over arrays. You can map over objects. You can map over observables. You can map over promises. You can map over trees. You can map – there’s literally thousands of things that you can map over. And realizing that all of these fragmented interfaces can be coalesced into a single interface. And so, it really is, I think the biggest thing is like the power of polymorphism for a function. Because that’s basically the problem that I think – it’s not I want to say basically the problem. I think it is a problem that a lot of the functional programming libraries suffer from, is that the only polymorphism is object polymorphism, which is kind of the native polymorphism in JavaScript. Whereas in systems like Haskell and like Clojure, you can have a function be polymorphic. And so, one function can operate on many different kinds of data, provided it has an interface. So, when we’re working in microstates, we use literally one function: map. The same, the actual same function, reference to the same function we use everywhere when we map. And we’re just mapping a bunch of different things.

So, I think that that’s one of the reasons that I prefer Rambda, for example, over Lodash, is because it has a form of polymorphism. Most of the things like maps and lenses and applicatives and stuff, almost everything works on both objects and arrays. That’s actually kind of nice. So, Rambda has a basic polymorphism. But I think one of the other things that is really empowering about Funcadelic is that it allows you to make the map function work on any data structure that you happen to come up with. Anything that you want. You can make it mappable and map will work on it.

TARAS: Yeah. I think for people, it’s probably quite abstract, what that actually looks like. I think one thing that’s interesting is that – so for listeners, the way that would look is you have a class, you have an ES6 class, which gives name to a certain piece of data that you have in your application. And then what you can do with Funcadelic, you can then say, “This is how you would implement – if you were to map this kind of an object, if you had an instance of this object, if you wanted to map that object, you can specify: what is the function you would use?” So, even though you would use the same – you would import map from Funcadelic and you would use that map to map whatever that object is and whatever type it is, but there’s a way for you to say, “For instances of this type, I would like to use this specific implementation of a map to map that object,” but use one function to do it. So, it’s going to use one function that you call. But you can specify, under the hood it can specify how the actual, what is actually used to map that instance.

CHARLES: Right. And then that’s nice, because then you can – anything that uses a mappable object, there’s a couple of reasons that that’s nice. Any time you can have some sort of higher-order mechanism that just requires that something be mappable, that it be a functor. And then it’s really nice because then you can have higher-order operations that they just need something that’s mappable. But you don’t have to use that one shot of actually having a map function as a property of the object. You can actually, you can kind of define wrapper classes or whatever, that then introduce a unique way that this object can be mapped. So, you can have the same structure and map it three different ways. Whereas you’re kind of constrained by that using normal OO inheritance because you have to have a map property on your object.

TARAS: Yeah. There’s something else actually, when you start thinking about this. For me personally, I think the first step when we start working with objects, having mappable objects, it was the first thing that was really helpful. But then I think really quickly, right after that, I think my second thing that I started using and I think is probably my favorite feature now, is append. I think it’s actually – yeah, so append is an implementation of a semigroup but I think it’s simpler. A simplest metaphor would be something like object assign. So, object assign is an implementation of a semigroup, except that object assign mutates the object that you pass into it. Right?

CHARLES: Right.

TARAS: It doesn’t create a new object for you.

CHARLES: Right. So again, getting to this idea that there are some universal operations. Like there’s this idea that you can take two things – I don’t want to say object because that’s an actual concrete type in JavaScript – two things and you can smoosh them together. And I think there was actually – wasn’t there literally a big controversy about this?

TARAS: Yes.

CHARLES: About like, array smoosh? I think this is nice – smooshing, right? But you can smoosh things together. And with addition, I can smoosh two integers together or two numbers together and get a third number, right? Or with objects, I can take two objects and smoosh them together by merging their keys. Or I can take two strings and I can smoosh them together and I can end up with a string that’s concatenated. Or I can take two arrays and smoosh them together and I’ve got now an array that’s been concatenated. So, what’s interesting is these are all very, very different types that I’ve just described. And yet, there is some fundamental operation about them.

And I think this is actually something that’s bad about JavaScript is there’s five different names for all those operations that we talked about, but it’s really one unifying concept. And that might seem like a small thing, but when you have five different interfaces for one fundamental construct, that leads to fragmentation and you can’t treat that data uniformly. And it ends up like, paper cuts, paper cuts everywhere. Whereas if you can unify all of these into a small, one thing, which is like we can append two objects, and then we’re going to have an implementation of append for array. And behind the scenes it’s going to call smoosh. And we’ve got a universal – we’ve got an implementation of append for object, which is going to assign the keys in an immutable way and return a new object. Or we’re going to have an implementation of append for string which is going to concatenate the strings.

You might even, I don’t know a hardcore FP nerd would have to probably correct me because this is just totally conjecture, although I know it’s probably a solved problem – is maybe you could say we append two functions together. And that returns a function which is composed, right? That might be a way that you could say – what would it look like? Is function a semigroup? Can we append two functions together? And maybe you end up with like a pipe or a function composition or something like that. But I think that highlights, when you have these universal interfaces, because literally I feel like most of the stuff that we have been working with, there’s literally five, there’s like five interfaces. And everything is one of these five interfaces.

And it kind of flips you on your head, because the classic programming wisdom that I have certainly have espoused for at least the last 10 years is that you don’t want to race to find abstractions. That’s dangerous. Because you can get locked into the wrong abstraction, right? Wait. Let the abstraction emerge. And I’m a lot less bullish on that concept now that I’ve discovered these things, because that wisdom is cultivated in a world where there are [billions] of abstractions, if you’re giving unique names to everything and the combinations between them. But if you are coming up in a world where there’s five basic abstractions, then it actually pays off to ask the question, “Is this thing a functor? Is it a semigroup? Is it a monad? What would that look like?” It’s a nice thought experiment that doesn’t require that much investment.

You can think about it for a couple of minutes. And usually, you can come up and say, “Yes, yes. It totally is,” and I can start using it. And now I’ve introduced this really powerful abstraction. Or you say like, “No, no it’s not.” And then that information is just as valuable. And so, it’s very low-cost to experiment with abstractions. And so, I kind of think of it as – I know I’m on a little bit of a rant here but this has kind of been a major revelation for me – is that when you have very few abstractions which you compose in myriad ways versus having a whole bunch of abstractions that can’t be composed very much, the cost for experimenting with abstraction and making the wrong abstraction is several orders of magnitude lower. And so, you don’t have to be as cautious. And you can actually use trying on abstractions as a tool, rather than a very, very high risky undertaking.

And just to kind of close that thought out, I think that – I don’t know if anybody else but me remembers the world before we decided we were going to make all of our web services RESTful – like when people first started building all these web services, we were just going crazy with the endpoints. And there was no rhyme or reason. Kind of weird arbitrary levels of nesting. Sometimes, you’d throw in an ID as a query param, sometimes you’d throw it in as part of the path. And then I definitely remember, it was probably around, I don’t know, 2010 for me where I listened to a podcast where James Edward Gray was talking about S3 and ‘Was it a RESTful interface?’ and the O’Reilly book. And it really clicked for me. And realizing that if you constrain yourself to thinking about your API at least as these fundamental operation of manipulating resources, and you were constrained to four verbs and everything, you want to have ID-based URLs and resources and as flat as possible, those constraints actually are very enabling for consumers of your API and for actually authoring an API. And I think it’s the same principle at work here. Anyway, so I’ll end that rant. [Laughs]

TARAS: Yeah. I think it’s, I think people could probably, for those who haven’t been in programming as long as Charles has been, it’s probably easier – Charles I think people could probably relate to what’s happening with components now, I’d imagine. Because having components essentially look the same across every framework, they all have props and they all render, return some DOM, or some variation of that. But it’s kind of the same thing. You take some data and then you return something that is going to become DOM. And I think having that as a rule for what a component is, you can then make really complicated applications using these fundamental building blocks. And then you don’t have to – there’s not really much thinking on, “Well, how am I? What interface is this component going to have?” Okay, well you know it’s going to accept props. And you know it’s going to render some DOM when you actually render it, right?

CHARLES: Right.

TARAS: That simplicity I think is really helpful. And I think it’s one of the things that – I think one of the things about Funcadelic that I really like is there’s kind of a really small set of rules that are really helpful. And these rules are actually, they make it predictable. Because one thing that I find really challenging with using Lodash or using Rambda is that because there are so many functions, it’s difficult to know what is actually going to happen when I do something. So, a good example would be like if you use omit from Lodash, Lodash omit will then, it will actually – one thing you can do is you can materialize your getters. So, if you have getters in your objects, those getters can become values on the new object that’s created. So, that’s one thing that could happen. Or if you use omit, your symbol, if you have values…

CHARLES: What does omit do, by the way? I’m actually…

TARAS: Omit is a pretty popular way to exclude functions. Basically like a filter equivalent for an object. It’s usually used to remove some props that are coming into a component. But it can do some – it can actually change the type of the object. One thing you know for sure is if you use something like omit or if you use assign, if you have an instance of something, guaranteed, working with that object in a mutable way is going to cause some really strange things. I think with omit, if you were to have an instance, it would definitely not give you an instance of the same type, like an object of the same type. It will give you just a regular [inaudible]. And there is no real way – you could create an instance. I don’t know what assign would do. I’m guessing that it would just take an instance and would put things on it.

But it’s really not – I think this kind of ambiguity doesn’t work very well when you’re trying to build something, when you really just need to know exactly how your tool behaves. I think that’s one thing that with Funcadelic, because it’s such a small API and because you know for a fact that the library is designed to be immutable and it’s designed to preserve type information, then you know that if you use one of the operations, you will get most likely the same kind of object. And you’re not – well, you will the same kind of object and it’s not going to mutate that object. And so, there are some of these guarantees that are actually really helpful and [inaudible] is liberating, I think.

CHARLES: Right.

TARAS: Especially when you’re trying to do more challenging things, not trivial, just copy some properties from one object to another. But when you’re actually doing more sophisticated things, in those use cases, having these kind of rules is extremely powerful.

CHARLES: Yeah. And I’ve definitely resisted and I think will continue to resist expanding the API surface area that much. Because it is, I think there’s only five or six functions in there. But what you get for those functions is extremely powerful and extremely predictable. I think it might help to give a concrete example. Like when you were talking about object assignment.

Like if you have a car class in your application and you want to do an immutable object assign, well the kind of stereotypical way to do that or the typical way to do that is you assign. You create an empty object and then you assign one car to it and then you assign the next car to it. And now you’ve merged these two car things, right? But then the problem is, you’re going to get an object out of that, not a car. It’s just going to be a vanilla object. Now, it’s going to have all the properties of both, but it’s not going to be a car. And that could be a problem if you’ve got any custom methods on the prototype, any computed properties on the prototype. It’s going to be a problem. Whereas Funcadelic, you can append two cars together and you’re going to know that it’s going to have both of the properties. You’re going to know that it’s going to be of type car. And you don’t have to worry about running the constructor or anything like that. It’s just going to have – the properties are going to be carried over properly.

If you’re using a library like Lodash or Rambda or something that doesn't account for type, because in order to append two things or map something, the implementation actually lives with the type, not with the function. The function is polymorphic, but the implementation lives with the type. You can then actually, you can always return the proper type. Because it’s ultimately – like if your map operation or your filter operation or your what have you operation doesn’t take type into account, then there’s no way to actually preserve type. But because we delegate in Funcadelic, it’s core to the concept. And so, it’s actually a very trivial thing to do, which is why you get that repeatability.

TARAS: One of my favorite things about append and semigroup implementation for object is that you can overload getters on an existing instance. So, what you get is you get a new instance with the getter that you passed to append applied to that instance. And this is kind of trippy but it’s actually really powerful because – so, let’s say you have an instance of an object and that instance has some getters on it. And those getters use some properties of this object to compute their value. And so, when you want – if I need to create a copy of that object in such a way that the getters still continue to work properly but I need to override how one specific getter works for one specific instance, one specific scenario, one specific use case in my code, then I can just use append. So, the first argument is the original instance, second argument is an object that has the getters that I want that I want to overload the getters on the original object. And then append will squish those things together, smoosh those things together, and give me a new instance that has the getters that I passed in, in the second argument. And all the same things that the first object had. And that object will work. This new object will be a fully-functional object just like the original object that I still have a copy of, that I can use.

But this is really interesting because one thing that I’m finding with having these kind of tools in my toolset is that I’ve had features that I needed to implement on a project. And the people that I work with are really technical. So, they know where problems are going to be. And so, the would write requirements for how something – for a feature that needs to be implemented. And knowing the problems, they’re like, “Oh, by the way. You’re going to have a problem in this area when implementing this kind of specific functionality.” And for me, I’m like, having this toolset, I’m like, “I don’t see it as a problem at all.” It’s so easy for me. Because I know that if I need to implement – so if I have something that has expected behavior, but I need to create something that behaves very similarly but slightly different in one particular use case, I can always just copy that object and then overload its behavior. And it’s still going to be a fully-functional object. And I think that alone is just not something that you can do usually. It’s not something that’s available.

CHARLES: Yeah. It’s a technique that I think was discovered. Maybe it’s not original to us. But it’s just, the tools enabled it in the sense that it’s like having a flashlight in a dark room. It’s like, that technique was always there. It’s just when it becomes so concise that it’s so easy to just append one more computed property to a thing, then you just wouldn’t have thought to do it otherwise.

TARAS: What’s interesting about this too, for me, is that – and this goes back to the context conversation that we had earlier – is that I think React brought into our lives functional programming kind of, in a big way. Because part of programming React applications is working with functional components and working with functional concepts. And a lot of the things, like Redux, a lot of these things are powered by functional programming. And they work together. They compose well together because they’re all from the same paradigm. But the problem is that there are all these concepts developed together over time. And they’ve been tested together and they’ve been formulated together in languages like Haskell. But the ideas that make all of these stuff work really well together, they haven’t really become available in the JavaScript community. And it feels like, it’s like we’re all using a language that we don’t fully understand. And it’s like we’re all – a lot of people in the JavaScript world who works, when it comes to functional programming, it’s kind of like having English as a second language. It’s like, you can use the words but you don’t understand the humor. And it’s kind of like, you can’t make a joke. It’s kind of like that. You can’t really express yourself correctly when you don’t have full grasp of the language. And I feel like how we use functional programming in JavaScript is a little bit like that.

And by starting to bring these ideas, moving the wealth of knowledge from the source into the realm where we actually need to use it now, we can actually start to take advantage – leverage all these insights that actually enables all of this. It’s not like – so, the idea of how to write React and think in a reactive way or think in a functional way, those things are not just owned by the React core team or they’re not owned by an elite group of developers who really understand how functional programming works. It’s just available to everyone. And all you need to do is just learn some of these concepts that glue all these ideas together that are fundamental pieces of how functional programming works.

CHARLES: Right. That’s a great point. And it points back to kind of the original reason that I wrote Funcadelic and then started and then continued to work on it with you, is that it really was – it was actually meant as a – it started out as an educational exercise. What would these things look like if they were translated into JavaScript? And it turned out that it rapidly became core to my workflow and way of thinking. And so, it really is, there are weird names to these universal concepts. It is true. It is a foreign language. I really like that analogy. But foreign languages sound weird, and when people are talking in a foreign language, you can feel excluded. And the really, the reason that we wrote Funcadelic and the reason it’s there is to make them accessible, these things accessible to you. So that those abstract foreign words can over time turn into concrete concepts that you’re completely comfortable with, just like any word in any language. At some point, you approached it having no idea what that sound represented.

And so, it really is trying to – the emphasis there is not to noodle about and dwell on the names of the concepts but to take the real things that you are actually doing, give them names and formalize them, to enable you to participate in this new functional world that you’re describing. Because I love that sentiment. It does not belong to the React core team. It doesn’t belong to an elite set of developers on this project or that project. It literally is a universal tool that is 100% achievable. And people don’t even realize how close – if they’ve been working with JavaScript for a couple of years, how close they actually are.

TARAS: Yeah. I think they’re really – for most people, if they were to read the readme and then – well, I think one of the problems, it kind of works with the language metaphor, is that you need an immersion, right? I think one of the reasons why Funcadelic really stuck and functors and semigroups and [filterable] and all of these things really stuck for me, and I’m thinking about how using monads and monadic operations or applicatives and all that stuff – the reason why it all stuck for me is because I’ve been able to talk to you about it. And I think for people, finding a network that will allow them to practice immersively, to think about functional programming, not just occasionally – I mean, you could do it occasionally as well, it just takes much longer. But once you really, once you have a few conversations where you try to dissect a problem in a functional way and you think about what these pieces are made of, it becomes very natural, very quickly.

CHARLES: Yeah. I think that’s actually a really great point. And the thing is, you can immerse yourself incrementally. So you can just say, “You know what? I’m just going to start using append.” Anywhere that I would concat two strings or I concat two arrays or I would do object assign – screw that. You can even just say, “Instead of using object.assign, I’m going to use append,” and start there. Or to say, “I’m just going to start with mapping.” I think also the thing that’s nice about it too, is the buy-in can be incremental. But you’re right. You do need immersion. You do need practice. You need to actually use, you need to use the functions and you need to be able to use them one at a time, so that your mind can close over them. So then, you can kind of move onto the next one.

TARAS: Yeah.

CHARLES: So, that might be one way, is to say – because you know, I don’t think I really understood. I was already using append and map ubiquitously before I really understand applicative/apply. So, you don’t have to grok it all at once. You can definitely bite it in chunks. And the best way to do that is to start with one concept and really just attack it mercilessly. And then also understand that there’s a natural sequence there.

TARAS: I would add a little bit of a caveat to that. I think there’s a thing about using – doing something for learning purposes and there’s another thing about shipping things. What’s interesting with Funcadelic and what’s interesting about a lot of these ideas from functional programming is that I think they give you benefits that you might have not previously thought. Like for example, if you’re going to concat two strings together, doing it with append is probably the most robust way of doing it, relative to just being able to use [inaudible].

CHARLES: Yeah, that’s true. You wouldn’t want to use a string [inaudible], wouldn't you?

TARAS: Yeah. But there are areas when using – there are times when things are just not possible otherwise. Like for example, if you wanted to treat an instance of a class in an immutable way, this is simply not possible in any way. So, if you’re going to say, “I’m going to work with a bunch of these instances of ES6. And I want to keep them as instances, because they have certain behaviors that I want to have. I want to have a getter, or I want to have a method on it. And I want to keep these things fully full instances, not broken, not be turned into objects. I want them to be normal instances but I want to work with them immutably. When I need to make a change, I want to get a new object and not modify that object.” So, if you set yourself that goal and you say, “This is what I’m going to do,” then you really are not left with very many options. You only really have, you have to use append from Funcadelic. And because alternatively, you’re going to implement something yourself. You might as well just use append. And [I think] that’s a good place.

I think if you’re starting to, if you need to make something lazy, if you need to delay an execution of something – so, instead of pushing that execution, instead of using object assign and then computing everything ahead of time, you can use append. And you can create a getter and you can delay the computation of that value until the point when the user actually reaches for that value. If you want to start doing that kind of stuff, you really are not left with very many options. And append is the way to do it. But that’s the thing, is when you start to set these kinds of standards for yourself, you level up in a way that is very significant. I think it’s like a natural progression of learning. You start off learning and anything goes, as long as you can make this website work, it’s like, “I’m happy.” And then over time you get better and better at it. And then when you get good at building applications, your next step might be like, “What if I was stricter about it? What if I could actually – what would that open up for me? What would that allow me to do?” I personally think about it that way.

CHARLES: Yeah. I think that’s a good way to think about it. You mentioned having a network for discussing these concepts and trying to internalize them. Let me first and foremost offer myself as somebody. If this is a hill that you are interested in climbing, and I think it is a very worthwhile hill to climb because of the perspective that you will gain from its summit, please reach out to me. You can contact me at cowboyd on Twitter or [email protected]. I’d be happy to discuss these kinds of things, because I think that these tools are just incredibly powerful and will improve you. So, if folks want to get in touch with you, Taras, where would that be?

TARAS: I’m [email protected] and tarasm on Twitter.

CHARLES: Alright. Well, thank you everybody for listening. And as always, if you want to get in contact with us at Frontside, you can just email us at [email protected] or give us a shout on Twitter at @TheFrontside. Thanks everybody. We’ll see you all next time for episode 100.

This Frontsider panel episode explores what virtues go into making quality software, such as having tests, making sure software is performant and accessible, and why you should try to avoid technical debt.

Transcript:

CHARLES: Hello everybody and welcome to The Frontside Podcast Episode 98. My name is Charles Lowell, developer here at The Frontside and your podcast host-in-training. With me today, we’re going to have a round table, a Frontside round table. With me today is Elrick.

ELRICK: Hey.

CHARLES: Joe.

JOE: How you doing?

CHARLES: And of course, Will.

WIL: Hello, hello.

CHARLES: Welcome, y’all. We’re going to be talking today about some of the things that we do around here, aside from trimming the shrubs and making coffee and snacking on Altoids. Like, way too many of them. Yeah. I was thinking we could talk a little bit about software qualities of relative things, like this software has these qualities. And I think that that kind of lofty goal of software quality is comprised of having a bunch of little qualities. The quality of having fewer bugs or the quality of having these things. And so, talking about all these things that we do and kind of what we do to make sure that we continue to do them. Or the ways that we can ensure that our software has these things. So yeah, we can just start really anywhere.

WIL: Yeah. So, one core thing is obviously tests.

CHARLES: That kind of falls under we want to have – really, there’s two qualities there that we want, right? Is we want to have…

WIL: Maintainable software.

CHARLES: We want it to be maintainable. We want it to be resilient to change. And we want it to work properly, right? Yeah, so we put tests in place to make sure that that happens.

JOE: Tests also inform design in a really positive way. A lot of the time, anyway.

WIL: Another thing that we like to include in our apps is responsiveness.

CHARLES: Yeah. And just making sure that you have – that it works on a multiplicity of devices, right?

WIL: Yeah. And not just the devices, but browsers as well.

CHARLES: Yeah. And it turns out it’s actually really hard to do that after the fact.

WIL: Right.

JOE: Yeah.

CHARLES: Making sure that lots of browsers, lots of devices. Because yeah, sometimes you have some weird screen width that is on some weird device, and making sure that that works. I guess there’s some overlap with testing there, too, isn’t it, right? Like you want to be running your tests on those devices at those resolutions to make sure that they’re going to work. This is something that we aspire to but I don’t think we’re quite there yet. It was making sure that our applications are accessible.

WIL: Yeah.

JOE: I’m very excited to learn more about this as we get into this, yeah.

CHARLES: Right, right. And asking the question, how is it that we actually can ensure our applications are accessible? We have very paved roads for making sure that our applications are resilient to change and that they have low bug rates and that they’re well-designed via testing. But what is the analog of testing for accessibility? What’s the way that you can put those guardrails in for accessibility? I have no idea. And that’s an ongoing conversation here at Frontside.

JOE: So, I guess I’m curious as to what technologies are actually involved in accessing a web application in – would it be reasonable to say a non-traditional way? I know there’s such things as screen readers, but is that all we’re talking about? Or what is the ecosystem that we have to consider supporting?

CHARLES: I’m certainly not an expert on this. We’d have to get Rob in here to chew our ears off this.

JOE: Yes.

CHARLES: But from what I’ve picked up from him and from our conventions with Marcy Sutton and some other folks that we’ve had on the podcast, it’s a big umbrella. So, it’s anyone using an application in a non-traditional way. So, whether that can have to do with limited vision, hearing, movement, range of movement, cognitive ability, it’s a gigantic whale of a domain.

WIL: Yeah. The topic of accessibility can definitely be several podcasts on its own.

CHARLES: Yeah. One thing that we’ve talked about is it would be great if you could drive your test suite through a screen reader or something like that. What would that even look like? There are a couple of open source ones out there, but they’re Windows-only. I think it was NVDA was the big one. And then you have a screen reader that then drives the applications in your operating system, so it’s going to vary per operating system. So, making sure that it’s accessible on Windows, at least as I understand it, is very different from making sure that it’s accessible on a Mac.

JOE: Yeah, it’s like a whole other layer. And it’s like BrowserStack outside of the browser.

CHARLES: Right.

WIL: There are things that you can do from the beginning that will make it easier when you get to that point. It’s just like using semantic HTML, knowing when and how to use proper aria labels. All these things, if you do it from the beginning, it’s not as big of a task as bolting it on afterwards.

CHARLES: Right. And I think we do have a leg up when it comes to web applications. It’s within our power to change. There are cross-platform of those technologies. But as you said, it’s important to put them in from the beginning. Because as we’ve seen, for each one of those categories, you’re accumulating debt if you don’t address it. So, there’s technical debt. But I think that technical debt can [inaudible] into a bunch of different areas. So, there’s technical debt in terms of the internal quality of your architecture, the way your software components talk to each other. And I think that that’s what people mostly think of when they talk about technical debt. But I think in terms of responsiveness debt, there’s a slice of the technical debt pie that has to do with making your application responsive. And so, if you don’t address making your application responsive, you’re accumulating debt and you might not know it. And if you’re not making your application accessible, then from the beginning you’re accumulating debt. So that if you have to go and try and figure out your accessibility story six months, a year, two years, you might actually uncover and say, “Whoops. I’ve been swiping the accessibility credit card. And holy crap, with all this. All my fines and penalties and compounded interest. Now I’m accessibility bankrupt.” And that can be scary, right?

WIL: Yeah. And a lot of people don’t realize with all this debt after the fact is they think they’re going in and adding things like responsiveness and accessibility and tests. But really, you’re also taking away previous work that’s already there, things that need to be refactored. If you put these things off, you’re not just adding a few hours of time. You’re inflating your time exponentially.

CHARLES: Right. Right, exactly. It can be intimidating but I think it’s also empowering, because technical debt is like a scary subject. But if you’re like, “Oh, we can actually slice our technical debt into a bunch of different categories and address them individually,” just knowing that this is an area where debt can accumulate, that’s half the battle. Because the worst thing is debt you don’t even see.

ELRICK: Yeah.

WIL: I mean, [inaudible] is big. That’s a big part of accessibility, that, is most people don’t think of accessibility. So, that is a huge debt that a lot of companies don’t see.

JOE: What about something like internationalization where I feel like I’ve never been in an application where that wasn’t punted on to some degree. That’s kind of a well-known problem, but it still takes a back burner. Do you think that if accessibility had more exposure as a concern, would it actually get the attention it deserves or is it kind of destined to, “Oh, we’ll get to those yaml files later. We’ll send those off for translation later,” that type of thing.

ELRICK: I don’t know. Sometimes I feel as though people feel as though they’re trading speed away when they’re building applications when they go to implement these things. Like, “Okay, well we’re not really going to touch on these right now because that’s going to slow us down from pushing out features.” Which is not really true. Because if you don’t settle on these things early, you’re not really building a solid foundation for your application in the long haul. So, I think people are like, “Oh, we’ll just do it later.”

CHARLES: Right.

ELRICK: And, “We’ll just ship features now.”

CHARLES: Right. I think that’s exactly right. It has this kind of secondary effect where not only do you develop the debt but you develop a culture of accumulating debt, right? Like when it comes to people getting a hold of their finances, the first thing that they have to change is they have to change their spending habits. And that can be the hardest thing. It’s not just balancing the equation. It’s like saying, “I need to readjust my thinking about this.”

ELRICK: Yup.

CHARLES: So that I’m not consistently put in this situation again.

JOE: So, there’s an operative word there, right, in personal finance in that usually if a company is addressing technical debt especially down the road, something that they’ve punted on for a while, it’s far from personal. There’s a board of directors or there’s a special interest group involved. There’s people who want features that are putting money into it. There’s a lot of pressure as the company grows and more people are involved. Priorities are more likely to be lost, I guess.

CHARLES: Are you saying it can be hard when your culture is spread over that many people, it can be hard to shift?

JOE: Absolutely, yeah. And I guess to keep with the dash-first thing, ideally were we starting a company, we would want to start a culture for this company. A culture that recognizes the vulnerability that we all have to technical debt as applications grow. We want that upfront. But the reality is, you know, startups are eager to get things out. Companies that have been around for a long time have high-paying clients that they depend on that want certain things. And yeah, I guess I’m just saying that it has to come in from the beginning.

CHARLES: Yeah. And I think that – I don’t want to completely disparage technical debt entirely, because technical debt like actual debt, like financial debt, is a powerful tool that you can wield. But it’s also, it’s like a table saw. You can also easily slice your finger off. It doesn’t mean that it’s not a useful tool, right? If anyone’s bought a house, it’s really great that you can borrow money to buy a house. It’s great that businesses can borrow money and get small business loans to get bootstrapped. And that benefits us all to have that community. I don’t think that – yeah, startups definitely, they need to have technical debt as a tool that’s available to them. But they just need to understand the consequences of it and be able to get a hold on it.

JOE: That’s a super interesting take. I never considered it that way before.

CHARLES: Yeah. It’s definitely not my take. I actually think the person who coined the term ‘technical debt’, that was the original idea. But then people realized that technical debt can also get way out of hand.

WIL: It’s just like real debt. If you’re not paying down a certain amount every so often, it’s going to keep growing.

CHARLES: Yup. You’re going to have to declare bankruptcy at some point and throw out the piece of software if you don’t pay a down. And that’s going to be more expensive.

ELRICK: Yup. That’s definitely true. So, I have a question. And we see this all the time repeating itself at various companies, whether it’s a startup, a large company, where they put off testing and mobile-first, user-first, accessibility-first. Like all the firsts, they just toss it to the side. Why do you all think that that happens so frequently?

CHARLES: I think it comes into people not understanding that if you don’t address it from the start, it won’t happen naturally. There is a prime motivator that has to happen. If you don’t imbue something with those qualities when it’s tiny, when it’s a tiny seed, a tiny crystal, you’re going to have to drill through layers and layers and layers of core to put it at the crystal to begin with. I like to think of software as kind of like a tree. And we eat the fruit of the tree, and that’s the features that users use. And we can tell that a fruit is delicious merely by placing it in our mouths. And we can tell what fruit is bad. But we can’t really look at the fruit itself to say what caused this fruit to be good, what caused this fruit to be bad. We have to look at the tree. And I think that that’s what people miss when they’re developing software, is that what you really want to do is you want to build a tree that builds good fruit. You can’t just take the fruit off the vine and say like, “Hey, I’ve got this peach but it doesn’t have enough sweetness. So, I’m going to take a syringe and I’m going to inject glucose around it and make it less tart.” You say, “I want a sweet fruit,” right?

ELRICK: Yeah.

JOE: You could probably actually do that.

CHARLES: You could. And that might be a strategy. And we see a lot of software that has those qualities of, “Oh, we’re going to make this accessible,” or, “We’re going to try and make this beautiful.” I happen to think that pigs are adorable animals and look great in lipstick. But that [laughs]… you could put lipstick on a pig but people can tell. And you can say, “Oh, this peach needs to have softer fruit,” and you can whack it with a mallet to actually make the meat more tender. But people are going to be able to tell. So, what you really need to do is you need to care for the peach tree rather than worry so much about the fruit. Because if you have a healthy tree, then you will have healthy fruit, right?

ELRICK: Yeah. So, you want to plant good seeds.

JOE: Yeah.

WIL: Back to you question, Elrick, about what motivates startups and other companies to put off these things. I think the biggest thing is just time and money. They have this misconception where they’re saving a little time and saving a little money now just to add it back later. But in reality, it’s going to cost them tenfold time and money for adding it later, versus just spending that little bit of time and money and all that to begin with.

CHARLES: That’s true.

JOE: It could also boil down, as far as just personal intimidation. Not so much like a business side of a thing but maybe just, think of all of the things that you listed, Elrick. It was almost a dozen dash-firsts in there. If you’re sitting down at a startup that you started with three friends and just approaching these things for the first time, that’s a lot to tack on right upfront. It’s intimidating.

CHARLES: It is intimidating. I think my message to those people is I’ve felt intimidated by that. I think my message to those people is like, the nice thing about it is if you attack those, if you tack all of those things from the get go, the features will take care of themselves and feel more effortless as you go on. You say like, “Oh, well actually, I don’t worry about a high rate of bugs.” I want to say recidivism, but that’s not the right word. A high rate of return, not on money but on – or high rate of bouncing your users. You don’t want that. And if you bake that in from the beginning, parts of the software development cycle that were stressful before just aren’t stressful anymore. So, if we say, “We want to have a system that is easily maintainable, well let’s put that in from the very beginning.” We say that a lot. We deploy to production on day one. But what that means is, we say we have this value that we want the system to be easily maintainable. And so, we’re going to do it from day one. That means that we actually – it’s not something that we worry about so much on down the road. Whereas that used to be very stressful. I don’t know. I remember when I started my career, there were these long release cycles where every six months, you’d release software. And the last month was just absolutely terrible as you try to stand this thing up and get it into production and then realize it’s not monitored. There’s no one checking the health of this thing. So, it’s pissing off users at one in the morning. And…

WIL: Beepers.

CHARLES: What’s that?

WIL: Beepers.

CHARLES: That’s actually a great – there’s a story there. The one time I got a beeper, I went canoeing in the canals of London and I tipped over my canoe and I dropped both my cellphone and the beeper that they’ve given me.

ELRICK: What?

CHARLES: I never got put on pager duty again.

[Laughter]

JOE: I’m going to use that next time [inaudible] with an on-call position. That’s a good move.

CHARLES: I remember, I definitely remember how sour my manager’s face was when I turned [inaudible] the cellphone that was like, dripping with water.

JOE: He was eating bad fruit, probably.

CHARLES: Yeah. [Laughs]

So, the other thing is we like to build beautiful applications, right? So, you have to – that match the user experience. You have to spend that time on design and beauty upfront. You will not have a beautiful application after the fact. You just need to bake it in.

ELRICK: And accessible design.

CHARLES: Exactly.

ELRICK: Don’t forget that one.

CHARLES: Don’t forget that, right? A responsive design.

WIL: Yeah, accessibility-first in design. Yeah, responsive and all that starts in design phase, yeah.

CHARLES: Yeah, all that, right? So, you want a great experience. You want an accessible experience. You want a responsive experience. You want a quality experience. You want a performant experience. That’s another quality that you say. Like, “We’re going to make sure that this is performant.” If you want that – and that’s something that we’re not always great about, right? We don’t actually put in benchmarks for our software from the get go. But maybe we should. But there’s perhaps a hidden cost there that we might be actually accumulating performance debt that we don’t even know about.

JOE: That’s true.

ELRICK: Interesting.

JOE: So, things that pop up that are new. Like, accessibility wasn’t probably always a thing in computing. Internationalization probably wasn’t always a concern. Beautiful certainly wasn’t a concern if you look on Wayback machine. You will see that to be true, right?

[Laughter]

JOE: So, all code is tech debt, I would argue. Or at least has the potential to be. And yeah, as the ecosystem as a whole evolves, being responsive to that, having plasticity in that respect, sort of like meta-first.

CHARLES: Right.

JOE: That could be the real challenge.

WIL: Yeah, Charles is mentioning all these experience things. And so, I was thinking X-first is simply experience-first. You want you users to experience a certain quality of your app. That experience needs to start in the conception phase.

CHARLES: Yeah.

ELRICK: That’s true. And even your developers coming in, developer experience.

JOE: Yeah.

CHARLES: Right. And I think the core of that X-first, that experience-first, is you need to pick which experiences. Because you can’t have everything.

JOE: Right, yeah.

CHARLES: One, there is going to be too much. You have to say, “I’m going to sacrifice on knowing that this is a performance thing. I’m not going to include that in the core DNA of my application.” And there’s just going to be things that you don’t know about yet that are just unsolvable problems or that don’t necessarily work. And you can say, “You know what? Hypothetically, I’m not going to make this an accessible – I’m not going to focus on accessibility.” But then you need to own that. And you need to know that you’re accumulating a huge amount of debt around that. And then I think that is a particularly bad trade-off because someone’s always going to come along and you’re going to have to know that your application is accessible. I think once we clamp down on that, that’s going to be something that we have a strategy for and we include at the beginning on every single application, right?

ELRICK: Yeah.

CHARLES: But I think you need to have, almost like holding the cards in your hand, say, “These are the cards. These are the X’s that I’m going to have in my hand. And they are going to be core to my app.” And they’re going to be part of the DNA of that tree. So that I know that the fruit is then going to have those qualities.

JOE: And then you as an engineer, that goes through an iterative process as well. Just starting out, you have no idea what that DNA should look like. And short of learning from people who are wiser than you who are around you, and reading blog posts and whatnot, really the only way to know the pain of strong-arming internationalization for instance into a 15-year-old Perl application, is to go through it. And then, you know, future trees will not have this DNA.

CHARLES: Right. Right. And that’s the other thing. Is if you are going to include, if you are going to try and splice something into the DNA, there’s a lot of work. And you just need to go for it. You acknowledge that it’s going to be a lot of work. And you need to, you just need to own it and go for it. And pay that expense of actually getting it deep, deep, deep into your application’s core values. So that then, you don’t have to worry about it anymore. Otherwise, you’re going to be paying – you’re just basically signing up for a lifetime of debt. Right?

WIL: Yeah. And then to make the debt analogy even more, it’s like people don’t understand the total debt. The end debt. People get a $30,000 loan with a 4% interest and they think they’re paying back that $30,000 loan. But really, they’re paying back $36,400 after all the amortization of their interest. The debt is higher than you can see, always.

CHARLES: Right.

WIL: And it’s true in tech debt, too. React is the new hot thing now, but in 10 years we’re going to be on React debt that we’re migrating away from.

JOE: I hope so.

[Laughter]

CHARLES: Maybe less, I think less than 10.

WIL: Yeah. The debt is always there. And people don’t realize how much they have to pay on top of what’s visible.

JOE: Yeah. It’s an invisible vig.

CHARLES: What’s a vig?

JOE: It’s interest, in the mafia.

CHARLES: Oh.

JOE: Sorry. Yeah.

CHARLES: I forgot you’re Italian.

JOE: Yeah.

ELRICK: So, for people that are listening, they might be in a situation where they need to advocate to the powers that may be these X-first values. What do you all think that some of the approaches that they should take to say to whomever it is that, “We need to do this first”? Because there’s times where you might say, “Hey, we need to do this first,” and people just look and say, “Oh, maybe not.” Then you need to push back on that.

CHARLES: In my experience, I find that the tech debt argument is a good one. Because I think it can be, it’s both limiting and empowering. Because sometimes it really is the right call to pull out your credit card and put something on it. If you need to buy water and you need to buy food and you don’t have any other means, man, put it on the credit card. Right? Seriously. Even if you have no idea how you’re going to pay it back. Like, whip that sucker out and stick the chip in. And it doesn’t matter how much it costs. And so, sometimes that is the right call. But I think draining it of a moral or a value as a human person thing, and approaching it from a business decision and saying, really trying to attach a cost to it. Because then I think if you can drain out the emotion of it, because people really want something. They’re striving to go get it and trying, give them tools to think about it rationally. That I think is a good strategy, to just say, let them know that there is a debt that’s being paid here or that’s being accumulated here. And it’s really large. And maybe even say, “Look, if we were to put this off by six months, this might cost not twice as much. It might cost ten or even a hundred times as much.” So, by saving $5,000 now, you might actually be accumulating $50,000 worth of debt. It’s [bigger] than you think.

But I do like – so, I think that’s one important tool. But I think then also the other important tool is to say, “If we are going to attack this, let’s drive it home. Let’s put it at the core. Let’s make this a value that we hold so that the tree can take care of the fruit itself.” So, if we say that we’re going to put in accessibility – because not all projects are greenfields.

JOE: Absolutely not.

CHARLES: So, what’s the message to them? Sorry. You’re just SOL. I think if you’re a year into a project, two years into a project, and you realize, “Oh no. We need to do internationalization,” recognize that that might be something that’s – that’s a pillar of your architecture. Or, “We’re going to make this application accessible,” don’t half-ass it.

WIL: Weave it in.

CHARLES: Say, “We’re going to transform this. We’re not going to add accessibility. We’re going to transform what we have into an accessible application.” Or, “We’re going to transform what we have into a beautiful application.” Otherwise…

WIL: Yeah [inaudible].

CHARLES: I would say leave it ugly and focus your efforts elsewhere on things where you do have your values straight. Because you’re never going to have everything in line.

JOE: No.

WIL: Treat software like immutably. You don’t add something to it. When you want to add accessibility, you’re creating a whole new accessible app.

ELRICK: Ooh. That’s deep.

CHARLES: Yeah.

JOE: So, having seen – I don’t know. I think it was very apt, looking at it as a business decision. I’ve seen it go the other way. Because at least among engineers and people on the technical side of it, this can become a very strong moral issue that people feel very strongly about.

CHARLES: Because we have to live with the consequences quite honestly, right?

JOE: Exactly. And that’s a hard thing to translate to say an executive board that may be three levels abstracted away from you and is making those decisions. I’ve seen people attack or approach this I guess with that emotion built in, with the, “This is the right way to do it. Everybody else is doing it wrong.” It gets nowhere, basically. What needs to happen I think, so you talk about having this beautiful tree. But that also requires beautiful gardeners. And so, where the moral thing or the interpersonal thing comes in is there needs to be kind of an inclusive and encouraging environment that is fostered among the people tending to the tree. And that’s a totally separate thing than selling the business value of it. Those things should be completely divorced.

CHARLES: Yeah. It’s funny. It’s always hard to reconcile those two things, right? Because on one you have, “You have to take care of the raw consumption of material and the output of product.” But then also trying to – so, there’s some baseline math that has to happen but making sure that that goal, it doesn’t slice people. And can enable them to be happy and feel like they’re doing good work. And that the things that they’re doing is having meaning. It’s probably an insoluble problem that we’re going to be dancing around for as long as people are around. If there’s one thing that we’ve come to recognize around here, and we’ve stated it many different ways from a bunch of different angles through the course of this conversation, and I would say through the course of this podcast, but that is if you want to see something in your software, make sure that you attack it from the get go.

ELRICK: Intertwine it in your DNA.

CHARLES: Exactly. And then you can actually experience the fruit, rather than trying to always, always trying to jam it and change it and get it into the taste you want after the fact.

So, I guess that’s it. Thank you so much y’all, for this conversation. I really, really enjoyed it. For those of y’all listening, if you want to continue the conversation, you can get in touch with us we are @TheFrontside on Twitter. Or you can drop us an email. We’re [email protected]. So, thanks Elrick. Thanks, Joe. Thanks, Will.

JOE: Thank you.

WIL: Thank you.

ELRICK: Yup. It was great.

JOE: It was fruitful.

[Chuckles]

ELRICK: Frontside-first.

CHARLES: And well, we’ll see y’all around.

Erich Gamma: @ErichGamme

Show Notes

Resources

Transcript

CHARLES: Hello everybody and welcome to The Frontside Podcast Episode 97. My name is Charles Lowell. I’m a developer here at The Frontside and your podcast host-in-training. And with me today, we have two very special guests. They have been working on technologies that have run very parallel to my entire career as a software developer. And we’re going to talk about that. So with us today are Erich Gamma and Dirk Baeumer who are developers on the team developing VS Code, which if you’re in the frontend space is taking that area of development by storm. It’s just amazing, some of the things they can do. Lots of people are using it every day. Lots of people are trying it. And so, we’re going to talk about the technologies that underlie that and the story of how it came to be. So, welcome Erich and welcome Dirk.

ERICH: Hello from Zurich.

CHARLES: Alright. Zurich to Albuquerque. Here we go. As a first start, I would have to say my first contact with this story, I at least have to mention it because – and this is for Erich – you wrote a book that was very, very instrumental in my formation as a young developer. I think I was about 22 years old when I read ‘Design Patterns’. And I don’t know. I still carry a lot of those things with me to this day, even though a lot of things have changed about the way that we do development. I still carry a lot of those lessons, I think especially things like the state pattern and the strategy pattern, and stuff like that. I want to move onto other things, but I was hoping that we could talk just a little bit about, what are the things that you find still kind of relevant today?

ERICH: Well, now as you said, some of the things are kind of timeless and we’re lucky to have found these things. And I still love all the patterns. But I must say, things have changed, right? So, at that time, we thought objects are very cool. And as we have evolved, all of a sudden we think, “Oh, functions are actually very cool, too,” right? Closures and so on. So, I think we got more broader and of course if you use functional programming, you have many more patterns available as you program. So, I feel some of the object thinking still applies. But that’s not the only thing that counts anymore. Today it’s functions, stateless, immutability, and all those things within functional programming which is [straight] and which [inaudible] in our team.

CHARLES: Yeah, yeah. I would love to see an update to how do these concepts transfer into functional programming. But anyway, just wanted to say thank you for that. And it was about the same time that, a few years after, I don’t know the exact same timing, I want to wind back. Because we’re going to talk about VS Code but before VS Code, there was a project that both of you all worked on called Eclipse, which I also used. Because at the very beginning of my career, I did a lot of Java development. And it really opened my eyes into a level of what tooling could do for you that I didn’t see before. And I was wondering how did you arrive to there? Because before that, I was using Emacs and Vim and Joe’s Editor and things that were editing the text files. And how did you kind of arrive at that problem? Because I feel like it’s very similar to the one that VS Code solves, but this was what, 15 years ago?

ERICH: I think it’s older, right?

CHARLES: Really?

DIRK: It’s 17, 18, yeah. Yeah, yeah. It was end of the millennium, right? So to be honest, Eclipse wasn’t the first development tool we worked on. Then, we worked on the company ObjectTechnologyNational. They worked on Smalltalk tools. And of course, Smalltalk had a great IDE experience, right? So back then, Java became popular. One idea was, how can you preserve the great Smalltalk coding experience? [Inaudible]

CHARLES: Ah, okay.

DIRK: [Inaudible] and find all references, method-level history versioning, and so on. So, that was the input that got Eclipse kicked off. And one idea we had at that time, Eclipse is our opportunity to make everything right. And as we have seen now, when we did VS Code, we could even improve what we have [inaudible] at that time.

So an example, in Eclipse we thought plugins are very cool and we have kind of a microkernel. And you load all of the plugins in the same process, they have a rich API, and so on, which is great. But we found over time, if you have lots of plugins and they do bad things and they run in the same process, it’s not the best thing.

CHARLES: Ah. Right. And so…

DIRK: [Inaudible] have a different architecture. We believe now in isolation, separation. So, we now run extensions in separate process that communicates through RPC with the IDE so that we are in full control. And we can always say you can save the tool, save the document, no matter how bad a plugin behaves and decides to do an endless loop. Because in a separate process, the hope is still one CPU is open, available for you, that it can be safe from the other process. So, that’s some example, right? Eclipse has done many things right, but the multi-process architecture I think is a major switch. And the other major switch is at Eclipse time you think Java is cool. Everything has to be in Java.

CHARLES: Right.

DIRK: No longer think like that, and that brings up this other topic of then the language servers that we can also talk at some point.

CHARLES: Right, because that’s the thing, is VS Code – now I’ve primarily been exposed to it through JavaScript and TypeScript development. But it really, it’s designed to support all kinds of different languages. So, the C++ support is really good. The C support is really good. And I assume the Java support is really good. Is it safe to say? Because I only ever used Eclipse in the context of Java. Did Eclipse gain kind of a wider acceptance further beyond Java and C++?

ERICH: Yeah. I think it’s fair to say Eclipse has a rich ecosystem. Yeah. I think with all the tools. And it will be interesting to see that you can close the loop, because for Visual Studio Code, when you do Java development, you actually run Eclipse behind the scenes. That’s how we kind of smiled at each other, Dirk and I, when he said, “Now we close the loop.” We started with all JavaScript and then we integrate Eclipse using this language server protocol and that’s how we close the loop.

CHARLES: Ah.

DIRK: So maybe one thing I would like to add is that when you look at Eclipse and the tool and framework landscape that existed in the Java time, at that point in time when we started with Eclipse, it was very well-defined. There was Java. It was a well-defined set of libraries you were using and frameworks you were using. And if you look at the programming and tool landscape you have today, in months you see a new framework for JavaScript popping up or there’s something else or another cool X, Y, Z thing. So, the tooling you build today has to be a lot more open to these new inventions, especially since they occur in a higher frequency than they did in the past. And that had influence on how we architected Visual Studio Code to give people a lower barrier of integrating their stuff into Visual Studio Code than you typically have in Eclipse.

In Eclipse you needed to program in Java. With the LSP you can program in any programming language. In Eclipse, if you really want to try to do something nice with code complete and stuff like that, you had to hook up a lot of stuff. So, we raised that to another abstraction layer where we more talked about what people provide on data and we do a lot more for them in the user interface than compared for example to Eclipse, which lowers the barrier for people to integrate languages in Visual Studio Code than the barrier you had to integrate something in Eclipse. And so, [inaudible] for that one was that there are a lot more tools and programming languages out there that have importance than 10 years ago.

ERICH: I’ll give you an example. So, when we did C support in Eclipse, and it was also the team that seeded it. Of course, it took over and has now a great community behind it in Eclipse. But you wrote the C tooling in Java. And of course, that means you built the parser in Java and then of course, there are great C parsers around, C frameworks. But also it means you cannot dogfood what you write. You write Java but you don’t program in C++. I think which is what makes VS Code so appealing is we are a very aggressive dogfooder. We want to use ourself and of course [inaudible]. That’s why [inaudible] is very good. The C++ guide, they programmed C++ and they write in C++ so that’s how they make it very good, that you have this feedback loop.

CHARLES: And so, what’s an example? We’ve talked about this low barrier of entry. So, if I were wanting to say, I do mostly programming in JavaScript. Let’s say I wanted to add, I know all of this already exists, this infrastructure already exists, but let’s say I wanted to add smart editing to JavaScript source files. What would that process look like for me as a JavaScript developer?

DIRK: To be fair, whenever it comes to language services, it’s never easy. But [inaudible] lower the bar. A language always means you have to do parsing, you have to do [9:59], type bindings. You have to make it fast, scale high up, and so on. So, this is never easy. But I think if you think about the different steps you can do, the first thing, let’s not take JavaScript. Let’s take a new language.

CHARLES: Okay.

DIRK: Your new cool language.

CHARLES: Or maybe we take a Lisp or something where writing the parser is very easy.

DIRK: Even that, you have to resolve symbols and so on.

CHARLES: Okay, okay.

DIRK: Even the parsing [inaudible]. But yeah, let’s take a fancy language like Lisp or whatever. So, the first level I think is you want to get some nice coloring. That’s the first level.

CHARLES: Yes.

DIRK: So, you get some coloring. And what we do there actually in VS Code is we tap into the community from TextMate. So, we use TextMate grammars to support colors in languages, which gives us access to a long [10:51 tail] of languages. So, to change the [10:54], if your language is not too exotic, you will find the grammar that describes how to color, what the tokens are in your language, and then you can get your language colored. That’s step one. The next step is of course you want to get smarts like IntelliSense and so on. Ideally of course you can say, “Well, maybe there is something already around that has abstracted the parser and you can use this library.”

CHARLES: Right. Because there actually are a bunch of JavaScript parsers written in JavaScript. I know I keep coming back to JavaScript, but let’s assume with this language that we’ve got. I may not have to write a parser but I’ve got one.

ERICH: You’ve got one, exactly. You’ve got one, right, and then technically it’s not in the same language as the tool. So, that’s why I don’t want to go too much into JavaScript because for instance VS Code is written in TypeScript, which [transpiles] to JavaScript, which moves a little bit, makes it not as convincing as it could be. So, let’s say it’s a different language. Your fancy language is written, has a parser in your fancy language, which is different than the language of VS Code which is JavaScript.

CHARLES: Right.

ERICH: So, then the next level is to say, “Okay, well you have your code you encapsulate it in a server that you can talk to through some protocol.” And now the challenge is what protocol do you talk to? Typically in the language, the library you get, it will use some ASTs, symbols, type bindings. And what Dirk mentioned with lowering the bar is that assuming you have those ASTs, the way you talk then with our tool is through a protocol that is not at the level of the ASTs but at a higher level.

CHARLES: A higher level than the ASTs.

ERICH: No, yeah. A higher or simpler level. Let’s give you an example. You want to find the definition of a symbol in your fancy language. The way the protocol works is you only tell it, in this document with the URI, at this position, I want to find the definition of the symbol that is this position. The request goes over the wire to the other process. Document URI, and the textual position. And what comes back of course now in the server you used AST, you find the symbol, you find the binding of the symbol which means it gives a definition for it. Of course you use your AST to analyze it. But then what gets back to send over the wire is yet another document, the reference, and the position.

CHARLES: I see. So, you’re really like pinpointing a point in just the raw bytes of the document. And you’re saying, “Look, what is here?” And you just want to delegate that completely and totally to this other process. So, the IDE itself doesn’t know anything about the document?

ERICH: It knows about the document, right?

CHARLES: I mean, it knows about the textual positions of the documents and the stream of characters, but not the meaning.

DIRK: True. The smarts are in the server. And you talk to the smarts at the level of documents and positions. And the [good thing is] it’s a protocol, is at this level it makes it easy to integrate into one editor, which is VS Code, but also into other editors. So, that’s why we came up with the idea to have a common language server protocol which allows to provide a language not only for one editor but also for many editors. That was a challenge we had in VS Code. Remember when we started, we were kind of late to the game. We said, “VS Code should be in between an IDE and an editor.” But what we liked from an IDE is of course code understanding, IntelliSense. Go to definition, find all references. But how do you get that for a long tail of languages? We cannot do it all ourselves. So, we need to get a community to tap into. [Similar to] like TextMate grammars are kind of a lingua franca for coloring. So, we are looking for the lingua franca for language smarts. And that’s what the language server protocol is, which means you can integrate it in different IDEs and once you’ve written a language server you can reuse it.

CHARLES: I guess I’ve got two questions. What are the kind of things that I can do with a server that implements the language server protocol? And then I guess the – so we’ve talked about being able to find a reference. And is there a way you can incrementally implement certain parts of the protocol as you go along?

ERICH: Yeah.

DIRK: Yeah, basically you can. The protocol on the server and the client side talks about capabilities. The server can for example say, “I am only supporting code complete and go to definition and find all references.” And for example, something like, “Implementation hierarchy or document symbols or outline view is not supported.” And then the client adapts dynamically to the capabilities of the server.

CHARLES: Okay.

DIRK: That’s one thing. And the set of capabilities is not fixed. So, we add them. We just added four or five new capabilities to the protocol last week. So of course, we listen to requests that come from other IDEs, what they would like to see in the protocols that we see in Visual Studio Code, we would like to extend. And that’s the way we move the protocol forward.

CHARLES: Okay.

DIRK: It’s capability-based and not so to speak version-based. So, [inaudible] versioning at the end of today.

CHARLES: Right. You can incrementally say, “I’m going to have,” if I’m starting to write a server, I can say, “Well, I’m going to only start with just find definition at point.” And that’s the only thing that my server can do.

ERICH: Well, there are some basics, right? Keep in mind you have two processes. And once the user opens an editor, the truth is in the buffering memory on the one process. The basic thing you have to in a language, so you have to support the synchronization of [inaudible]. Once you open a file in the editor, then the truth in the buffer, and then you have to sync it over.

CHARLES: Right.

ERICH: [Inaudible] close the truth on the file system and you also have to tell this to the server. Because the server has to know where the truth is.

DIRK: That’s correct. These two open/close handshake methods and change methods, this is the minimum you have to implement. But for example, for Node itself, we provide libraries that help you with this. And the protocol is not very complicated. It’s a buffer. Then it’s change events. Either it’s an insert, a delete, or an edit.

CHARLES: So, let me try and get this straight in my head. I think I understand. The problem is that the VS Code, or your code editor, it’s actually making changes to the buffer, and it needs to communicate those changes to the server. Or does the server actually make the changes itself?

DIRK: The editor does make the changes. So, the protocol is spec’d in a way that as soon as an editor opens a document, the ownership travels from the server for the content to the tool. And the server is basically not allowed to read the state of that content from disk anymore, or get it [inaudible].

CHARLES: Aha.

DIRK: Therefore, the client guarantees that everything the user does in that document is notified to the server, so that the server can move the document forward.

CHARLES: Okay.

DIRK: [Inaudible] we see the close event, that basically with the close, transfers the ownership of the document back to the language server. And it is allowed to re-read that content from disk if it wants.

CHARLES: Okay.

ERICH: Here, the protocol is really data-driven. Dirk mentioned that earlier, right? So, basically what flows between the server and the tool is data. So, what do we mean by data? You ask for IntelliSense or completions at the line. What follows is just the data. A list of completions that flows then from the server to the client. And then the client decides what to do with this data and decides to modify the document by inserting the completion proposed that the user selected.

CHARLES: Right. And then if it decides to make any updates, it needs to send those to the server.

DIRK: Exactly.

CHARLES: So, if I actually insert the method that I want to call there, I’m going to be inserting nine characters, and I need to tell the server, “Hey, I just inserted nine characters to this document,” something like that?

ERICH: Exactly.

CHARLES: Okay. And so now how, because I remember now one of the coolest things about the class of tools of Eclipse that I hadn’t really seen in the more lightweight editors – I went from Java, like so many of my generation went from Java to Ruby and then to JavaScript – once I moved out of the Java world, one of the things that I had come to expect from my tools was that they would help me make modifications to my codebase at a very high level. So, I would be able, if I had some class that was imported into say five modules in my codebase, I could say, “I want to change the name of this class,” and then it would find the references and then make the updates to those things. So, how do you manage that? So, if I have a class called ‘Person’ that I want to change to ‘User’, if I change it to ‘User’ then it’s going to break in those five different places unless I rename it to ‘User’. That’s something that was very doable in the Java world. How do you keep the code editor, the tool I guess is what you were calling it, in sync? Like the server is going to make that change or does it just come back with data and says, “Here’s the references if you wanted it to change”?

ERICH: Yeah, yeah, yeah. So, two things you mentioned, right? Java and JavaScript or course. Java is a typed language which means you have better understanding of the code and what the reference is. In JavaScript which is typeless, you cannot know it as much, so that’s actually why we developed also, we’re using TypeScript. VS Code is actually [written] in TypeScript which allows you to do these kinds of things like refactorings. But if you look at the language server protocol, it has support for rename. And the way how rename is done is again it just documents positions. You say, “At this position, I want to rename the symbol with this other name.” And then you tell this to server and the server will handle the rename by giving you back a list of positions that need to be updated.

CHARLES: Ah, okay. So now, I’m starting to understand what you’re talking about when you say data-driven. It’s literally just telling the tool – the tool proposes, “I want to do this rename.” And then the server provides all of the information that is required to actually do the rename. But it doesn’t actually do the rename itself. It just provides the data.

DIRK: A couple of reasons for it. The data effects, at the end of day, it’s again, edit, and it’s more or less the same edits the client sends to the server when the user types in the document. This is the protocol. On top of it, something that you can create a file or rename a file, this comes as a result back to the client. And then there, since it is a client/server architecture, the whole process is async. So, we have to give the client the change to revalidate if that edit structure that comes is still valid. If it is still valid, the client basically applies it. And by applying these edits to these documents, they will automatically flow back to the server until the client either closes these documents again or saves them. So, the reason being is that some of the tools may even show you a preview. You can only select some of them and apply them. So, there’s always an interaction in these refactorings and to make that possible, as Erich mentioned, the whole protocol is data-driven. We don’t go the server and say, “Okay, do that rename,” and he writes that back to disk. It computes a set of transformations to bring the current state of the workspace into that new state after the refactoring.

CHARLES: I see.

ERICH: [Inaudible] be fully transparent. Actually, no. Refactorings, Dirk [inaudible] refactorings for Eclipse so we can go deep on that. What we don’t support right now in the protocol, we support edits in the buffer but when you want to rename a class in Java, you also want to rename the file. And that’s something we’re currently working on to support in the specification of the language server protocol. So, we don’t have that yet. But we support code actions, quick fixes, that you like from Eclipse probably. And you can use then to do refactorings like extract method, extract constant or extract local variable, things like that you can do at the level of the language server protocol.

CHARLES: Wow. That is…

ERICH: I think [inaudible] right now. Let me go back to the Java thing. The Java language server actually has the support for refactorings. And there is now a language server protocol implementation of this Java provided by Eclipse. So, all the support you had in Eclipse for Java or most of the support is now also enabled in VS Code.

CHARLES: Right.

ERICH: [We don’t] really have to reimplement it because you can reuse. And that’s the big thought we have. You want to reuse language smarts as much as possible because they are so hard to implement.

CHARLES: Right. And so, you can do that because you’re providing this abstraction between the tool and the actual smarts, which is really, really cool. I do have to… how do you make it fast? Because you’re describing this tool, this client and this server, and they’re syncing. They’re keeping this distributed state in sync and you know, how do you keep that from coming too chatty? Or is it something that you have to consider? Or is it just, maybe I’m overthinking it because I haven’t dealt with it?

DIRK: So, at the end of the day, it is chatty. But it is made performant in the way that it’s very incremental and partly event-based. So for example, if you type in the document in the editor, you can either decide to [inaudible] sync the full content of the document, which we do not recommend but for some basic exploration, that is something people do. And we have [inaudible] the delta-encoded mechanism. So, we sync the buffer once and then after that you only get the edits the user does. These are chatty of course since the user types them, we debounce them and collapse them on the client side and only send them if we know that the server really needs to know them because we have another request we are asking the server or after a certain timeout. So, there are smarts behind it. But the protocol is kept performant by making it an incremental protocol at the end of the day, and not sending too much data back and forth.

ERICH: Right. We don’t serialize ASTs. We serialize positions, a list of items for completions. And actually, the transport is just JSON RPC.

CHARLES: Okay.

ERICH: And actually, someone, there is different usage now for language server protocol. And there is one host, Eclipse J, which brings it again back to Eclipse. They actually run language servers remote.

CHARLES: Interesting.

ERICH: And if you use it, you can run it on the browser, you get IntelliSense, and of course I guess it depends on how far away you are from the server. But it seems to work, according to feedback we’ve heard.

CHARLES: Really?

ERICH: The feedback we heard from them [is pleasant]. So, they use many of the language servers.

CHARLES: So, is this a product that they have where the language server is running in the cloud and you send – your entire codebase essentially goes over to the language server and you can export the smarts to the cloud?

ERICH: It’s one step at a time. So, Eclipse J is kind of, they have what they call cloud workspace, which means the workspace is in the cloud. And [inaudible] code smarts of the workspace in the cloud, they can run the language servers in the cloud. It’s a [inaudible]. One user has one workspace, has one language server.

CHARLES: That sounds amazing. And if they can make it performant.

ERICH: We have done cloud IDEs, right? If you look at the history from Visual Studio Code, you also had our stuff running in the cloud at some point. That’s how we started. Before we pivoted to VS Code, we built – our exploration was, that’s why the project is six years old. The first two years, we explored how far you can get coding done in the browser.

CHARLES: Right.

ERICH: And we had some [inaudible] there.

CHARLES: So, I’ve played around with a lot of cloud IDEs and I’ve found them to be neat, because every few years it comes along. But yeah, it does seem that there are certain challenges that it’s nice to have a client running and just be able to have the files locally. And is that a performance thing or if VS Code is written in TypeScript, theoretically it could run in a browser, right?

ERICH: Of course. The [inaudible] there still runs in the browser. Then it’s used by many tools that run in the browser. Like actually, if you want to edit your source code in the browser, there it’s using the same editor that’s running VS Code. So, that’s how we started. Cloud IDEs, yeah we were at this point. We had our cloud IDE. We could edit websites in the browser, source control them, have a command line, deploy them. What we found is it’s great for some scenarios like code reviews or doing small tweaks to files. But when it comes to really development, you use so many other tools. And you want to just have them. And [inaudible] a long tool chain problem. So, as a developer, you just want to use other tools as well. And that’s why you can’t have them all in the cloud.

CHARLES: Right.

ERICH: And [inaudible] we said at some point, it was a great lesson we had that you can program in the browser. But now we want to go to have a really [seven by 24] coding, you want to have a desktop experience. So, what we then did, we moved over the code we had run in the browser using a shell, the Electron shell, and can run it on the desktop.

CHARLES: But there’s theoretically, you could be running your language server for example in the cloud, but everything else on the desktop.

ERICH: Yeah. Some people do that.

DIRK: Right.

CHARLES: Okay. Wow. It’s crazy. It’s heady stuff. We’ve talked about the barrier to implement the code smarts is much lower than it has been in the past. What kind of proliferation of code smart tools are there now that implement the language server protocol? Like how many different languages would you say have airtight…?

DIRK: So now, [inaudible] time where we don’t count anymore. You tell us a language and I can look it up, whether it’s supported. Tell me a language and I can tell you whether – no, we have a website.

CHARLES: Okay.

DIRK: And when I look at it, we have about 40 languages.

CHARLES: Wow. That’s probably about, pretty much every mainstream language.

DIRK: Yeah. I cannot find what isn’t there.

CHARLES: Yeah. It almost kind of begs the question, is this going to be the new bar for a language? Because I remember when I was starting out, really you just needed to have some interpreter or some compiler to have “a language”. And nowadays, it’s not just the language. You need to have a command line tool for managing your dependencies. And you need to have a package system with a public repository where people can publish reusable units of code. And what’s become expected out of a language to succeed has upped. Is having a language server implementation going to be part of the bar, the new bar, for “Hey, I’m thinking about creating a language”? I haven’t really arrived until I have a package manager, I have a command line for resolving dependencies, I have documentation, and I have a language server.

DIRK: I personally think that is our dream at the end of the day, to get there. We know about languages that do so. So, a lot of these language servers come for example from the people that developed the language. For example, the WASP guys, they do the compiler and they actively work on their language server as well. So, at the end of the day, the advantage of that approach since the WASP language server is written in WASP and runs in WASP, they can reuse so much code that they already have written in WASP. That’s easy for them to package that up in the server and basically the people that maintain the compiler, at least the same team, maintains the language server at the end of the day.

ERICH: And that’s why we call [those] a win-win for the language provider. Because if you implement the language server using the language server protocol, then it can be integrated easily by the tool provider. And it’s a win for the tool provider since there is a common protocol across all these languages you have to support. You can write an implementation once and again benefit and support many different languages, which makes the matrix problem one language support for each tool into more a vector, right? It reduces the matrix into a vector. You only write language servers that get integrated into different tools.

CHARLES: Right.

DIRK: And [inaudible] especially I think appealing for new languages that come out, because it lowers the bar for them to get into existing tools. Because if they write a language server speaking the language server protocol integrating that at the end of the day in Visual Studio Code is basically packaging up an extension for Visual Studio Code and writing 20 lines of code.

CHARLES: Yeah.

DIRK: And same [inaudible] for other IDEs that exist where people implemented the language protocol client side for the tool, for example. For vim or for Atom.

CHARLES: Yeah.

DIRK: So, new languages I think definitely, we see that trend go onto the language server protocol because that gives them an entry point into a large tool community.

CHARLES: Yeah. I’m really excited about it. I’m actually an Emacs user. And that’s actually how I found out about LSP, was in my Emacs newsfeed I saw that someone was starting on LSP support, and got digging into it. And I think that one of the problems that has plagued not only Emacs but all these editors is what you’re describing where for example the JavaScript support was really great – is really great – in Emacs. There’s refactorings. There’s IntelliSense, code completion, all that stuff. But that’s because someone wrote an entire JavaScript parser and code smart system in Elisp, which is just an absurd hurdle to jump over, to expected. And so, what you expect out of your editing experience, like when I went to try – if I were to go to try Python, well it’s not nearly as good as what I’m expecting. And so yeah, I think it’s exciting to hear what you’re describing where with having some shared set of abstractions, you can offload all of that code smart onto the community that’s building these new tools so that they’re really easy to integrate into your environment. I think it’s really exciting.

Although it does make me ask – and I think we’ve got time for one more question – is we’ve been talking about all these different languages. Java, C++, JavaScript, TypeScript, Ruby, Python, et cetera, all these, the 40 languages that you talked about that have this implementation. What are the challenges that you’ve encountered trying to build abstractions that work for 40 different languages? All with their different syntax, all with their different conventions. It sounds like when aside from the fact that you’ve actually done it, I would say it’s impossible. So, I’m curious. What were the unique challenges to solve there?

DIRK: I think we already touched that at the beginning, the appealing stuff of the LSP is that it’s not talking about the programming language itself. It’s talking about things I can do with source code. For example, requesting code complete, go to definition, find all references. And the data that flows between the client and the server is not in terms of the programming language itself. It’s about editor abstractions. We talk about documents and positions. We talk about edits that are applied to documents. We talk about snippets and stuff like that. And these abstractions, since they are programing-language-neutral, are a lot easier to implement for different editors. And the [inaudible] where the [inaudible] would speak AST nodes and symbols and functions and classes and methods, that at the end of the day, would not work. Because if I ask go to definition, the result is not a function or a variable definition. It’s simply a position in the document with a hint which range to select.

CHARLES: Okay. Yeah.

ERICH: [Inaudible] places. In only a few places, we have to really abstract across languages. Like for instance, completions. When you do completions, you don’t know, is it a variable? Is it a function or a method? That’s where we have to abstract. But that’s one of the few places. But again, it’s an enumeration.

DIRK: Yeah. And that’s only to present an [icon].

ERICH: Yes.

DIRK: It’s only to give you a nice icon in front, because when you insert it, what comes back for completion item is basically a textual edit or a bunch of textual edits that when you select that completion item, we take these edits and apply them to the document buffer. And whether you edit a functional programming language or some other stuff, Prolog or whatsoever, it does not matter at the end of the day.

CHARLES: Yeah. That simplicity, and treating it at that simple of a level is what unlocks all those superpowers.

ERICH: It unlocks lowering the bar. But of course, if you look at some [of the demands], refactorings, whatever, they cannot easily be funneled. Not all of them can funnel to this low-level abstraction. Then of course, the criticism of the LSP protocol is that if you have already a very rich language service, you might not get it all through the LSP.

DIRK: That’s true.

ERICH: And the [inaudible], that criticism we see of the LSP. But it’s a tradeoff, like so many things in software.

DIRK: Yeah. But what we learned there looking at different types of refactorings, it’s more the set of input parameters that vary much between languages. The result of a refactoring can for every programming language that is at least document-based, [inaudible] in that lingo the LSP speaks. Because at the end of the day, it’s textual edits to a document, right?

ERICH: So, many people like LSP but there are people that don’t like it. And people that have rich language services like IntelliJ, [Cool Tool], and [inaudible], even with LSP we would only get 20% of [our cool] features. Which is a little bit downgraded and not really true. But you see, it’s a tradeoff.

CHARLES: Right.

ERICH: And if you want to [inaudible] language available broadly, I highly recommend it packaged as a language server. Your chances that it gets used, supported by different tools, is much higher than anything else.

CHARLES: Right, right. So, it’s kind of like, what’s the UNIX thing? The universal text interface and how it seems counterintuitive but it actually just means you can literally compose anything. Because so few assumptions are made.

ERICH: I would just recommend, [inaudible], go to the website that we have about the language server protocol. I’m pretty sure it will be in the introduction or whatever. It’s microsoft.github.io/language-server-protocol and then you see the implementations, all the implementation of languages, who integrates language servers, and also what kind of libraries are available, if you want to implement your language server.

DIRK: And a full specification.

ERICH: And the specification is there as well. Yeah.

CHARLES: Yeah. If you want to go ahead and do it yourself. Well, thank you so much, Erich. Thank you so much, Dirk, for coming on the show to talk about the language server protocol. It’s very exciting to me and I think it’s exciting for development in general because I just think by having – even if it’s 20, 30, 50% code smarts for ever single language, just the billions and billions of hours that you are going to save developers over the next, over the coming years, it’s a great feeling to think about. So, thank you for all your work and thank you for coming on the show.

ERICH: You’re welcome.

DIRK: Yeah. It was fun talking to you.

ERICH: Yeah. [Inaudible]

CHARLES: Yeah. If people want to continue the conversation, is there a good way that they can get in touch with you?

DIRK: Usually GitHub Issues. So, where the language protocol is, it’s a project on GitHub. Simply find issues. We accept pull requests. I think that’s the way we communicate.

CHARLES: Awesome. Again, if you want to get in touch with us, you can get in touch with us at [email protected] or you can reach out to us on Twitter. We’re @TheFrontside. So, thank you everybody for listening. And we will see you next time.

Show Notes:

Resources:

Transcript:

CHARLES: Hello everybody and welcome to The Frontside Podcast episode 96. My name is Charles, I'm a developer here at the Frontside. And your podcast host in-training with me today is Jeffrey.

JEFFREY: Hello.

CHARLES: Hello, Jeffrey. And Arash.

ARASH: Good morning.

CHARLES: Yeah, how are you doing?

ARASH: Doing great. Thank you.

CHARLES: All right. So, today this is going to be a little bit of an internal powwow where we're just going to talk about some of the patterns that we see as we develop software and just kind of a little bit of a chat on the way we do it. So, we're going to be talking today, I guess we could do like a little bit of a spoiler and just kind of co lead with what we're talking about.

ARASH: Which is...

CHARLES: Which is...

ARASH: Drum roll...

CHARLES: Outside-In development. And it's kind of the way that we've just naturally gravitated towards, towards developing software here.

ARASH: So first question is why we gravitated toward that.

CHARLES: I could just tell you kind of our personal history on this. Is that, I guess it was, would you say like around 2007, something like that kind of the pattern of like REST services got like really well established.

ARASH: Sure, that's accurate enough.

CHARLES: Yeah. It's like around mid-2000's. And so people just went kind of, you know, cuckoo for Cocoa Puffs except APIs instead of Cocoa Puffs. So it was like all the talk was about your API, what's going to be your API, and how are you going to present this to the world? And oh my goodness, you're opening up your system for extension by developers and it's really fantastic. So, it kind of became the norm, I feel, from that point forward to be thinking about what is the API that I'm going to be presenting rather than what's the application.

ARASH: That's your starting point.

CHARLES: Yeah, that's your starting point. So a lot of development and getting like a lot of frameworks came along to make API development really simple, regardless of run time. It was really just really simple to just set up API. And so, a lot of people did that and I think there was value in that and that you're thinking kind of about your external domain model, like how you're going to present yourself to the world. And so you really are focused on the constraints and the abstractions that you want to present and how you want to hide all the messy complexity behind your system. But the problem that you get into is in those days, especially on the web, the clients were very closely aligned with the APIs. If you're coming like from a Rails backend, you almost had one for one between your pages and your API.

If I've got a user's end point, I'm going to have a user's management page. And I think as clients became more stateful and more of the kind of interactions were ported over to the client, that synchronization and one to one mapping between what your API look like and what your interface look like, that started to crumble and disintegrate. And now I think that it's actually quite, I mean there are still clearly analogs because you're talking about the same entities. But your client is really now a full-featured application. And it's got complex rendering logic, it's got complex state management, it's got all these things that happened completely and totally separate from your API endpoints.

JEFFREY: And your UI ends up being aware of relationships between models. So yeah, it's just so much more sophisticated than what we used to need to do when everything was server-side rendered. When you're dealing with a heavy client side app, your API needs to be quite a bit more flexible really.

CHARLES: Right. And so what we were finding was that we'd often have to change the API significantly in order to support complex interaction on the client. But the problem is, is if you've started with your API, you've invested a lot of development, you've invested a lot of design and you've really laid down not just that design, but also the infrastructure and the operations to make it real, it can be very hard to change. And so, it can exclude a lot of the desirable interactions that you would like to have merely by virtue of the fact that it's set in stone. So I think that's kind of why we found ourselves...

JEFFREY: Why we discovered that inside out wasn't quite working for what we're doing anymore.

CHARLES: Yeah, exactly. It was like having a rock solid API was actually a drawback rather than an asset, whereas 10 years earlier it was an asset. And so it really, kind of having that insight made us step back and wind it back and say, "OK, where is the natural starting point? Where do we want to start?"

ARASH: So philosophically, what does Outside-In development mean to the Frontside?

JEFFREY: It means that instead of starting with the API, looking at actually the workflow is instead we start with what's the UI. What is the client side app going to look like and what are the needs of that client side app? And that drives the development of the API rather than solely, I guess you could say, the business models that would have previously been the initial driver for what that would look like.

CHARLES: Right, exactly. Letting the business models be kind of a function of the desired interaction or the desired experience and then letting it proceed from there. So really is yeah, it's starting from kind of what does the person using your system going to touch rather than what is the computer using your system going to touch.

ARASH: And so if one of our listeners is interested in Outside-in development, what kind of tenets would you recommend they follow and what are some of the best practices you can put into place to align well with this kind of thinking that we're talking about here?

CHARLES: I think that's a good question because there's obviously the philosophy of it, but then there's the practice and the implementation and what does that mean. And there's a lot. There's a lot of just kind of the way you approach the problem and then the tools that you're going to use. I don't know, Jeffrey, which ones should we tackle first?

JEFFREY: Let's tackle the tools. Why not? We'll go backwards. We probably should cover other things first, but why not?

CHARLES: We'll tackle the tools? All right, and this is actually a question I want to throw your way too is at what point do you...because you want to start really with having a good design, like a good wireframe? Do you prefer to start with say like something in InVision or Sketch or do you like to proceed straight to a working implementation?

JEFFREY: For me, it usually depends on how complicated of an app it's going to be. If it is something that's, I'm just dealing with like some crud operations, a lot of times I'll just go straight to the HTML and like starting to build that in actual JavaScript and we'll probably talk about the fake backend for that. But yeah, it versus something that's a little more complicated that has maybe a more novel approach or particularly interesting workflow or something that's going to be pretty difficult to code. That's when I'll start on in Sketcher, InVision, or a tool like that.

CHARLES: I think in the last project, we used Balsamiq. I mean, we don't want to get caught up on like tools, but I think that's a great tool because the other thing we did is it wasn't just one designer doing it. We actually had all of our developers going and writing Balsamiq sketches for the features that they were going to be implementing because it really means that they're going to actually put themselves in the shoes of someone using it and they're going to be thinking about what it's like to be a user. Because they've got to start drawing the boxes and lines and the buttons that people are going to click in and the text fields in which they're going to communicate with the system.

JEFFREY: And what was awesome about using a sketchy type of style like Balsamiq is that it removes the developers from thinking too much about having to make sure this headline is exactly the right font size, that I'm using all the exact right colors. Like, don't worry about that yet. Let's just get the wireframes in this sketchy style first and then we can work from there.

CHARLES: Because it really is. It's very valuable to have this process for yourself as one single developer to be able to view it from the outside in. And I think that looking at it one way, that's truly what it means to be full stack is that you can put yourself in the shoes of every piece of the system at each point. So I can put myself in the user's shoes, I can put myself in the client's shoes. When I say 'client', like the actual browser, the browser's shoes. I can put myself in the server shoes and I can have that perspective of each part.

JEFFREY: So that got me thinking about what browsers would wear what kind of shoes, but we'll leave that conversation for another time. Who'd wear the Chrome shoes?

ARASH: Chrome is definitely Nike, for sure. Speed.

JEFFREY: So back to the tool.

CHARLES: Just like Safari, like Hugo Boss.

ARASH: It's like, yeah, that's pretty close. Netscape is like K-Swiss or something like that.

CHARLES: It's got to be some defunct brand.

ARASH: BK Knights.

JEFFREY: Mosaic is the LA Lights.

CHARLES: Those were great.

JEFFREY: So once I have this design, I'm starting to move to code and I'm starting to build some UI, how can I iterate on my API from there? Because I'm starting to actually build a real app, I want to connect to something, I need a data store. What can I do there?

CHARLES: We've talked about this a lot, I think on this podcast, really over the years. And this is a technique that we got from the Ember community. There's a stack of fantastic libraries. There's Pretender which allows you to stub out XMLHttpRequests using a DSL, kind of like Node Express. And then based on top of that, there's basically an entire fake API layer which allows you to build, I would say it's not even really a stub at this point. It's like actually a prototype server implemented entirely in your browser. It's called Mirage.

JEFFREY: So, you think about it, almost like starting in Sketch and then going to HTML. You're starting in some pretty simple JavaScript and then you're working your way up to what that API design looks like.

CHARLES: Right. And so that lets you experience all of the nuance of API designs. So it comes with like you can experiment with different serialization formats, what are my property's going to look like? You can experiment with how do I load related data? So, if I've got a list of users and I want to get all of the comments that they've ever posted on my site, you might want to load those at the same time. So whether your API's going to support that and how it's going to support that, you can't really know until you actually have a consumer of that API. So what Mirage lets you do is it lets you consume an API you haven't written yet and you get to feel what it's like on the client and you get to make changes to that interface at a very, very low cost.

So there's no deployed infrastructure, there's no automatically generated documentation. There's no real consumers yet. There's no multiple clusters running in the Cloud and it's very, very, very lightweight. It's just running right there in the browser. And so what that lets you do is if you have an insight about the shape of the API, it lets you validate that insight. Or if you have an assumption that you have about your API that actually is going to be detrimental to your experience, you get to have that assumption invalidated very quickly. And so at the backend of this process, you end up with a backend that is optimally shaped to serve the experience that you're shooting for.

ARASH: So once you've got that, how do you reconcile it against is this a good API that I want to expose externally because maybe it's not.

CHARLES: Like what do you mean?

ARASH: Imagine that we've built the UI and we've iterated on the API in a fake stubbed way with Mirage or some similar tool and we've got to a state that we're pretty happy with, with the API design that works for this UI. Now, imagine we want that API to also be available to other things.

CHARLES: Right.

ARASH: How do we reconcile that?

CHARLES: That's actually, I think that's kind of an open question right now, right? I mean, we don't really have a good answer to it because it is a stubbed API. And ultimately you're going to have a real API. So at some point, you do have to say like, this is the shape that I want. I'm going to go ahead and create these things and I need to make sure that those two APIs actually line up. Is that what you're asking is like, how do I make sure that they line up?

ARASH: Sure. That's one piece of it.

CHARLES: Oh, that's one piece of it. OK, so this is actually...we'll explore this process. This is actually like a problem, I think, with the stubbed approach is you have your set of stubs and then you have kind of the real McCoy which eventually will come along. So, it's great to have a server running inside the browser, but at some point you're going to need an actual server that's running in the Cloud.

ARASH: Will you? Like what does the future hold?

CHARLES: I mean, yes because the data needs to move, right? The data needs to move off your laptop. Honestly, Arash asked a good point if you're making an offline application. So if you're making an offline application, this actually allows you to bring a persistence architecture to the browser that is like every other... you don't have to make a special one off case. You can treat your backend like a backend. It's just running on the frontend. So that's probably a corner case, but it's worth pointing out that there used to be like back in the day, if you're using something like Microsoft Word or Excel or some other offline app, you can go a long way without having any internet connectivity, but you get to build it in exactly the same way. And then when you are ready to have internet connectivity, you've got the architecture in place.

So, the future might not hold that and you might actually be able to use this in production. I would say it's a minority of cases. Most time you actually are going to want to have a server component. Let's say that you do, now you've got your real API and then you've got your prototype API that you run your tests against or that you're running against, and they need to line up because your frontend is going to be using both of them kind of throughout the development process.

So there are a couple of strategies to deal with this. I'd say one is you run automated tests against both versions and that's a whole another subject of having testable APIs. Like most APIs that we have that we develop aren't set up to do end to end testing including a lot of the ones that we've written, although I think going forward now that we have a better handle on it, we can do that. We can definitely unpack that subject at some later point like making testable APIs.

But I would say that the other strategy that you can use is use some sort of third party verification mechanism where you essentially record a bunch of interactions between your application and the fake API. And then you take those recorded applications and have them in some sort of repository and you can play them back on your real API and make sure that the responses generated by your real API for those recorded interactions to match up. So it's kind of like, I don't know, it's just a machine to make sure that the APIs stay in sync.

ARASH: That's a domain that we need to get better at.

CHARLES: Yeah.

ARASH: We're only scratching the surface there.

CHARLES: We're definitely only scratching the surface there. But the power of being able to develop without a real API first is enough so I think it makes the exploration warranted. There are some established tools in this space. The biggest one that I know of is called Pact. It's both a library and a repository. So it's kind of like got a...I don't know if there's a central server for it, but it's kind you record interactions, you upload them to a Pact server and then you can verify. So you can actually have a lot of consumers. And so, it's designed not only for making sure that your stubbed API works, but it's also for making sure that you just don't make breaking API changes. You can actually collect a lot of data about how people are using your API and then you can use that as a repository for when you make a change in your API to verify that'll work for all these other people.

So if I'm just some random person or some random developer using your API, I can...kind of like how you record [inaudible] statistics and stuff to make the Apple experience better or whatever. It's very common for developers to ask, to record for certain anonymous data. You can submit anonymous data in the form of like Pacts. And so that can help an API developer verify whether the change they're going to make is going to break clients out there.

So that's a little bit of a sidetrack there, but it's something that you start to think about a lot more when you start to develop in this fashion.

ARASH: Did you feel like Outside-In development is a timeless approach to software development?

CHARLES: I do. I feel like Outside-In, it actually even pervades more of software development than actually like the building the software. Like I feel that the longer you go in software development, you realize that the highest value activity is to front load understanding. So whether you're submitting a pull request, whether you're working on a work ticket, whether you're documenting code or even writing a method, the highest value activity that you can be engaged in, in each one of those points is understanding what the hell you're doing.

ARASH: And why, too. Right?

CHARLES: Yeah, exactly. Why? Understanding the motivation. What's your prime directive? What is the context on which you're entering into this piece of work?

JEFFREY: I'm going to go with a bit of a contrarian approach.

CHARLES: OK.

JEFFREY: I'm going to go with a "maybe"...

CHARLES: Maybe?

JEFFREY: Maybe this is timeless.

CHARLES: Maybe it's timeless?

JEFFREY: Because I think historically, so much of software development was working with constraints. And the particular types of software that we're working on, the most important constraints are on the frontend. What does the end-user experience look like? But historically, that hasn't been the case. Historically, the primary constraints have been what can this technology stack that I'm working on actually do?

CHARLES: Like what is the computer experience that I can support.

JEFFREY: Yeah. And so that was actually the bulk of the work was like how can I actually get something that I want out of this giant machine in a room with me? So, maybe it is timeless, maybe not. But I think it is the way going forward.

CHARLES: OK. So I agree that there's an interplay. There is kind of a yin and yang cycle. And I don't mean the cycle of we go to where we have to think about this and then we go to where we have to think about that. But I take your point, Jeffrey, but I think of something like Super Mario Brothers, which is I would say both a miracle of experience but also a miracle of technology in the sense that I think it was hand coded in Assembler on an 8-bit controller with who knows how much memory, the whole thing. So taking that into account, they had to think very strongly about what the computer could do at that point. They were operating under some just incredible constraints. But at the same time, they were thinking about what...they very clearly were focused on what is the coolest game that we can make at this point.

So you're absolutely right. You have to hold both inside your head. You don't want to go crazy. I mean, we could sit down on the Balsamiq thing and be like, the first thing we're going to do is we want a VR room with...okay, no. Let's dial it back.

JEFFREY: How do you wireframe VR? I've never seen anybody try do that.

CHARLES: The Frontside Podcast episode 97: Wireframing VR.

JEFFREY: Interesting.

CHARLES: How we wireframe VR. So yeah. I agree. I take your point. You do have to hold both in your head at the same time. But I would say start with one a little bit and then see where that can take you, that the task of gaining understanding about what you're trying to build [inaudible].

JEFFREY: Yeah.

CHARLES: If you try to understand what you're trying to accomplish, then you can try to push the tech stack towards that direction and maybe push a little progress forward. And then if you've accomplished something that you couldn't accomplish before, then you can start dreaming about even better experiences. And so kind of moving...like the wheels on He-Man's magic bus or whatever.

JEFFREY: This is out of my domain.

ARASH: I'm so [inaudible] backwards right now.

JEFFREY: Our references are not timeless.

CHARLES: Haven't you seen the...what's the one where he's like singing the song like the 4 Non Blondes song? And I said, hey, yeah, yeah. You haven't seen the He-Man, like the cover of 4 Non Blondes?

ARASH: Oh, like they dubbed his...

CHARLES: Yeah, no one's seen it? Oh, man.

ARASH: It's starting to sound familiar.

CHARLES: OK. I mean, this is like we're talking...

ARASH: This is like 2006 internet?

CHARLES: No, this is like 2010 internet. It's not the ancient past here.

ARASH: I'm going through my internet filing cabinet in my brain right now.

CHARLES: Okay. It had particular significance because the friends from attorney were also part of my childhood and I realized I'm unique in that aspect. But they basically, in those days they had the Hanna-Barbera cartoons were very cookie cutter, so they basically took the Mystery Machine from Scooby-Doo and they kind of re-colored it and they put like weird tank treads that like they put really funky kind of weird futuristic He-Man wheels on it. And it went from the Mystery Machine to He-Man's kind of a ride.

ARASH: They repurposed all the illustrations.

CHARLES: Exactly. Hey, you got to save time.

ARASH: Save time somewhere.

CHARLES: Doing your framework.

ARASH: Kids need cartoons.

CHARLES: That's right. So anyway, that's how the wheels on He-Man's Mystery Machine worked.

ARASH: This has been interesting for me because when we first started this conversation, we're looking at it as Outside-In development. But we've talked so much here about design and creation that it almost starts to feel like Outside-In development is almost too narrow. Like it's really what we're talking about here is Outside-In creation of anything useful and valuable to people.

CHARLES: Yeah. It's easy to overlook like whenever you want to create something, you get so focused on the actual building of it or the making of it happen that you kind of lose sight of where it is that you're going. And I think that sometimes it's important to be able to create without knowing where you're going to kind of push the paint around the canvas, so to speak. I think that's a valuable activity, like sometimes it's important to just code without really having any clear direction. You're just kind of following your instincts. Then it's the same way you look at Picasso's study in charcoal or whatever. There'll be like a picture of a horse leg and Picasso's like, "Oh, there's a horse leg. Man, how do I capture a horse leg?" And not really having much intent. And so that's important. But I think when you're building systems or you're creating something that you want to have a real lasting impact, you do need to consider it holistically and you really need to understand why it is that you're doing what you're doing.

JEFFREY: Part of what's awesome about working in software is that unlike Picasso who had to start a new piece, we can keep working on the same piece and be able to paint over it multiple times. And you can start with that outline of like, "Hey, this is the vision that we see this particular thing going." But then don't make space for like, "Let's just push the paint around and see what happens in this little section," and that's fine.

CHARLES: Yeah. And I think that because software is so malleable that it can end up capturing a lot of what is typically considered design. I think this has kind of also been a theme that we've talked about kind of over the last year or so is that over the last 20 years we've kind of seen software capture a lot of what were once separate practices. So when I was first starting out, it was quality. The line between QA and development was very stark certainly at the beginning of my career. And there was a fusion that was happening right as I joined of QA and development. And we saw basically the testing revolution, realizing the testing needs to be brought into the center of development, which needs to be put forth first if you want to have quality, be something that you want.

And then, a revolution that I saw kind of in the middle of my career is seeing operations. It used to be certainly for many years into the beginning of my career, like the people who maintain the software were very different from the people who wrote it and there was a big divide there. And with the Dev Ops movement, it's kind of the realization that no, if we want to have healthy operations, we need to understand what healthy operations are at the very beginning and we need to like put them at the beginning of the process. And I think that what we're seeing now, maybe a revolution that hasn't quite happened yet, but we're on the cusp of is seeing design burrow its way to the heart of development where it's like we want something to be beautiful. We want it to be frictionless. We want it to be delightful to its end users. It needs to be there from the get go and developers need to be thinking about it. It's not someone else's responsibility. It's you as the primary creator.

JEFFREY: It doesn't matter how elegant of an API you've created if the end user experience is just not there.

CHARLES: Yeah. So it's all about tearing down. There's been a series of walls that have come down that I've witnessed. And so I think this is one that's in the process of crumbling.

ARASH: So, this has been an awesome talk. How would you summarize Outside-In development and what it means to Frontside today and in the future?

CHARLES: I think that it really is about making sure that the understanding is solid. That's the core piece, frontloading understanding, realized through a stack of tools. So the tools we use are some sketch and design tool. We'll sketch out the idea we really like, ask questions and try and understand it and firm up in our idea, like have a vision of what the experience is going to be like.

Then the next step is to write some acceptance tests. So to define done, define what your criteria are going to be so that if I run these automated tests, the code will in fact realize the experience that I'm dreaming about. To do that, we've actually been assembling a suite of tools over the past year to do that and we've actually started releasing them to the world. So, it's still in a...I don't want to say alpha because we're using them in production systems, but it's much more of a work bench at this point than a framework, if that makes any sense.

So we have about four related tools under the big test tent. We have interaction library for stubbing out, gestures, mouse clicks, keyboard events, scrolling. We have assertion library that works with any other source. It's really an assertion extension or extension wrappers to make your assertions impervious to asynchrony. We've extracted Mirage from Ember CLI Mirage so that you can use it in any project, React, Vue, Ember, what have you. You can check it out at big test. It is for early adopters right now, but it is in production systems and if you want to acceptance test anything, command line, back client, what have you, if you're willing to write some JavaScript, you can write those acceptance tests and have them be robust. And whatever experience you're trying to create, you can validate it with that. And so, that's why we created it.

JEFFREY: And that's really a good marker of where we are in our experience with Outside-In development right now is we've been doing this in practice for awhile at this point and we're crafting tools to help us be even better at it and sharing those.

CHARLES: Right. And so we're pretty excited about these tools because I think they bring solutions to a lot of the problems you're going to encounter. So there's a lot more work to do and we're excited to do it. Head on, check those out. And we'll go ahead and wrap up. Just a few quick announcements.

If you're going to be at Assert(js) tomorrow, the Frontside is going to be there. I'm going to be there. Mr. Wil Wilsman is going to be there. So if you're there, please do give us a shout and we'll hang out. And then on the next podcast, we are really, really excited. I'm both nervous and I just can't keep a lid on it. We're going to have Erich Gamma on the podcast to talk about the language server protocol that he's been working on.

If you don't know Erich Gamma, he's one of the Gang of Four that wrote The Design Patterns Book. He was the primary architect behind Eclipse and most lately VS Code. So, if you've used any of those projects or benefited from them, which I have benefited immensely from all three, it's going to be really, really exciting. So, that's something to look forward to.

And with that, I will say goodbye to you, Jeffrey.

JEFFREY: Ciao!

CHARLES: And to you, Arash.

ARASH: Adios!

CHARLES: And everybody listening along at home or in your cars or cleaning your kitchen as I do when you listen to podcasts, we'll see you next time. If you want to get in touch with us, you can always give us a shout on Twitter, we're @thefrontside or you can drop us a line, [email protected].