The Frontside Podcast

In this internal episode, Charles and Wil talk about testing issues and BigTest solutions. Pieces of the testing story are discussed, such as the start and launch

Resources:

This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC.

Transcript:

CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode 115. My name is Charles Lowell, this episode's host and a developer here at the Frontside. With me today to talk some shop is Mr Wil Wilsman.

WIL: Hello.

CHARLES: Hello, Wil.

WIL: How's it going?

CHARLES: It's going good. I'm actually pretty excited to get to jump into this topic because we're going to be talking about some of the big things that are happening at Frontside and some of the things that we've been developing in almost for the last year.

WIL: Yeah. It's been about a year now.

CHARLES: It's been about a year and we've talked about it in various podcast but we're going to be talking about it again because there's just been so much progress that we've made, I think in a lot of clarity in kind of what we're going for here when we talk about BigTest and testing big and how we want to roll out the BigTest framework. We just have a lot more experience using it on a number of different projects, so we get to talk about that today.

Before we get started, I just wanted to talk a little bit about what BigTest is, both in terms of the framework and also the philosophy. Wil, you're the one who works the most on BigTest. When you think about philosophically, what does BigTest mean to you?

WIL: It's the size of your test, not a physical size like size and storage but how much your task actually does. The test itself can be very small as our test are but it tests the whole application from the user interacting with it down to the network requests. That's the definition of the philosophy of a BigTest to me. It's to tests your application from the biggest point of view.

CHARLES: Actually, achieving that can be surprisingly difficult, especially in a frontend JavaScript application and there are a lot of solutions out there for testing and we've talked about them. One of the questions that arises is when we talk about BigTest, what exactly are we talking about? Are we talking about a product that you can download and install? Are we talking about the philosophy that you just outlined? Or are we talking about the individual pieces of software that make that philosophy real? I think the answer is we're kind of talking about all three but we want to take this episode to talk about where we're going with the product.

What we've identified is the subcomponent pieces of that product. In other words, in order to get started testing big, what are the things that you need to think about? What are the things that you need to do? And then what are the component pieces? Because one of the things that I think is very important to us is that you be able to arrive at wherever you are in your project, whatever framework you are using, whatever current testing solution and be able to begin using BigTest. That means, you might be using some of it or you might be using a lot of it but we want to meet you exactly where you are, so that you can then, get onboarded and start testing big.

WIL: Yeah. Definitely an important distinction that we get confusion about is what is BigTests and people just assume like this whole test suite is BigTest but we used the parts of it ourselves like we use Mocha, which is not part of BigTest. We use Chai, which is not part of BigTest. We use Mirage which is kind of part of BigTest but definitely it originate in BigTest and Karma and things like that. BigTest isn't your testing suite. It's not one thing to go-to to grab, to start writing tests. It is a small pieces that you can use in conjunction with other small pieces, just to make it really easy and flexible to test your application.

CHARLES: Exactly. Because it turns out that there's a lot going on in the application. Maybe we should talk about what some of those pieces are that you might want to start using BigTest with or that you might need to test big, I guess I should say. What's a good place to start? Let's start with talking about some of the issues that you want to do when your testing big. Then we can talk about what pieces of the testing story that fit in to solve those issues.

One of them is you need to test that your application works, like actually works. That means you need to be able to test on a multiplicity of browsers, for example. We're limiting to the domain of web applications. There are actually a shockingly large number of browsers. It's not just Chrome. It's not just Safari. There's Mobile Chrome, Mobile Safari, which are subtly different. There's Edge and I'm sure the Mobile Edge is slightly different too, so you want to be able to test cross browser, right?

WIL: Yeah, absolutely and things like Nightmare and JS DOM and things that simulated browsers, we don't necessarily think those are the best tools for writing BigTest because we want to ensure that those browser quirks are caught and tested as well.

CHARLES: This is not theoretical like sometimes you'll have a syntax, like the parser is slightly different and you have something that throws a syntax error in Safari or in the Internet Explorer and your whole app is completely busted. If you just take in the time, just even trying to load the app in that browser, you would have caught that. That's what I've been on many times.

WIL: Yeah and what I just saw came up yesterday, which comes up frequently is not closing your CSS Selector and Chrome doesn't really care like web to browsers don't care too much but that will fail in Edge and depending on what you're missing, the failing is part of that too but mostly, Firefox and Chrome don't care about that kind of thing.

CHARLES: Right. It seems like the majority of testing solutions are kind of focused around Headless Chrome or some variation of Electron. That entire class of really dumb errors has already been caught. Like I said, to actually catch it, it takes less than a millisecond of CPU time just to load it onto the browser and see that thing doesn't work. Unfortunately, they can be catastrophic errors but the problem is how do you actually do.

We want to test like cross browser. This is something that we want to do. For me, I just can't imagine shipping an application without having some form of cross browser testing, some capability of being able to say, "I want to test it," like, "We want to work on these eight browsers and so we're going to test it on these eight browsers," but how do you actually go about doing that?

WIL: Right now, we are working on the BigTest CLI which will help us launch browsers but that's not complete yet. It has some bugs on. For the meantime we've been using Karma, which is great. Basically, you just have this service that's able to find the browser binary on the system and just launch them pointing to local hosts with your app loaded up and your normal development server take care of loading the test up and running the test. Karma and the BigTest CLI is just there to capture output and launch those separate browsers.

CHARLES: Yeah. I remember when I was first using working with Karma and I think Testim is another tool that's in this space. There's Testim, Karma and BigTest actually is we're developing a launcher because launching is something that you're going to need but it's such a weird problem. I feel like with the browser launchers, there's three levels of inversion of control because you're starting a server that then starts another process, which then calls back to your server, which then loads the app resources, which then loads the tests and then runs the test. There's a lot of sleight of hand that has to happen and –

WIL: Including injecting the adapter that you use, like the Mocha adapter, the Jasmine adapter that ends up reporting back to the CLI. That's something that Karma and Testim and BigTest will handle for you.

CHARLES: Right, so you're fanning out the test suite to a suite of browsers then collecting the results but basically, you need some sort of agent living inside the browser that's going to act on behalf of the test suite, to collect the results. I remember when I first came into contact with Karma and Testim, I was like, "This is so unnecessarily complex," but then, having used it for a while and I think there are some complexity that can be removed but if you want to do cross browser testing, that kind of level of ping-ponging is there's a certain amount of it that just necessary. It's something that's actually quite complex that you need to have in your stack, in your toolbox, if you want to truly test big.

WIL: Yeah and all the solutions is mechanisms for detecting when the browser has launched and restarting the browser based on its health check, etcetera and things like that that you wouldn't think of actually loading up a browser but you need to think of when you're doing automated testing.

CHARLES: What is it that sets apart, for example the launcher solution? We kind of call this class of solutions launchers, so Testim, Karma, the BigTest CLI. What is it that sets BigTest CLI apart from say, Karma and Testim?

WIL: We're trying to be as minimal config as possible and just really easy to get started and going. Karma has a lot of plugins that you need to make sure you have installed and loaded in the options set for those plugins. Testim has some stuff bundled but it still requires this big config bulk at the beginning that you need to passing or that's all what you were doing. We're trying to avoid that with BigTest CLI and one of the ways that we're able to avoid that is by just letting your Bundler handle bunding the test. In Karma, you need Karma webpack or something. Testim has some stuff that it needs and really, we just want like in-testing mode. When you're in the testing environment, just change your index to point your tests, instead of your application and your Bundler will do all the work and we just serve that file and collect the results.

CHARLES: Right, so it doesn't matter if you're using Parcel or you're using webpack or you're using Ember CI.

WIL: Yeah, Rollup even.

CHARLES: Or even just like low level Broccoli or Gulp or whatever. There's a preponderance of bundling solutions and that was always something that was just a huge pain in the butt with Karma. I know it's like just getting to the point where my tests are loaded and you look with Testim, most of my experience come with Testim comes through how it's used in Ember CLI like the histrionics that undertaken just to bundle all your tests assets and your application assets and your vendor assets and just kind of bootstrap that thing. It's a lot of work.

WIL: Another thing with BigTest CLI that doesn't include in Karma and Testim does is a concept of a watcher because all these Bundlers, you have HMR -- hot module reloading, Rollup and things like that. It come with plenty of plugins. Parcel is always set out of the box, so if you're using your Bundler, your existing Bundler to bundle your test, you get that watch feature for free, so it's another complexity that the BigTest CLI kind of eliminates.

CHARLES: What it means is we've hidden most of that complexity. Just let the Bundler handle it, right? The Bundler is the part of your project that bundles.

WIL: Yeah.

CHARLES: You should have your launcher actually doing that for you but we still do need to have some way to do that set up and tear down. When we have that testing endpoint, we have some way to say, "We're starting a test, not the application. We're ending the test, tear it down," so how do you abstract that away?

WIL: That's kind of something that we can't really avoid. It is just like some sort of dependency on the framework itself, your application framework. It's like you need to mount a React app. You need to mount an Ember app, etcetera and there's different ways to mount those things. This is one of the things that can't really be decoupled as much as everything else can but BigTest has BigTest React and BigTest Vue and we want to eventually gets like BigTest Ember but really, the main export of all these packages is just a simple mount helper that will mount and clean up your application for you and your testing hooks, whether you're using before each from Mocha or before from something else like Jasmine. You know, no matter what you're doing, you just have a hook that mounts your application and then, cleans it up on the next mount.

CHARLES: It's worth pointing out here that this is kind of a core concern of testing and testing big is being able to mount your application and tear it down with regularity and having hooks into that process. Whether you're using BigTest or whether you're not, can you still use BigTest React and BigTest Vue, even if you weren't using anything else?

WIL: Yeah, absolutely. Like I said, they just export simple mount helpers. I don't even think they have any other inner BigTest dependencies. They just have pure dependencies on their frameworks.

CHARLES: Right and so, you could use it, even if you wanted to roll everything else by hand or you wanted to get started somehow and you needed to do set up and tear down, again this is something that's key to being able to test big, so you should be able to use it independently, whether you use the CLI or not, whether you're using any of the other tools or not. All of the tools can be used independently.

WIL: Then another feature of the BigTest React and BigTest Vue is the tear down that happens before set up, rather than happening after your test runs, having a separate tear down. This allows it. Whether your test passes or fails, you can look at it and play with it and inspect it and debug it much easier than if you had tear down. You have to disable at tear down or throw a pause in there to keep other or something.

CHARLES: Yeah, I love that. When something goes wrong, you can just let the test case run and the last test that it runs, it just leaves at set up. It does the tear down right before the set up.

WIL: Exactly, yeah. At the very end of the whole test run, there's an app there waiting for you to play with.

CHARLES: If you focus in on a single test, we most commonly use Mocha, so you say a '.only' to run that single focus test, then you have the state of the application at that test case set up and ready to go. You can just play with it, you can inspect it, you can actually just use it as a starting off point and interact with the app normally as you would.

WIL: I want to say, Cypress does this too. They do their tear down before they're set up as well. That's how you're able to play with Cypress test.

CHARLES: Yeah, I like that trick. Now, we talked about launching, setup and tear down but we haven't actually talked about much of what actually happens in the test cases themselves. We talked about how to start and launch your test suite, how to do that across a bunch of different browsers, how inside of that, you have a separate concern as applications set up and tear down and how you want to lean on how you're actual app is actually bundled because that fits in with the philosophy of testing big. You don't want to use an external Bundler for your test suite. You want to use your real Bundler, how the asset is actually going to look.

But when it comes down to actually writing the tests, you need to be able to interact with at the highest level as you possibly can. When I say highest level, we want to verify that the users, when they take certain actions, we'll see certain outcomes and so, we want those outcomes and we already talked about this to be reflected in a real DOM, in a real browser. But at the same time, the real interactions, we want those to be as high fidelity as possible, so you want to be sending events to the browser. You want real mount events, real key events, real interactions.

WIL: Yeah, interacting with application. That's another core philosophy that we kind of talked about earlier that defines a BigTest. It's the user interacting with your application. We're not calling methods and expecting other callbacks or arguments to be passed or clicking on a button and expecting a message to pop up that says, "Form submitted successfully." These are user-facing things were starting on and acting on.

CHARLES: Yeah and then, it can be really tricky because these things don't happen synchronously. They're happening inside of your browser's event loop. I click that button and then it goes off and there's some loading state and then, I might get an error message that pops up this thing that animates out and then, goes away. The state of the browser is in constant flux. It's constantly changing and so, it can be very difficult to put your finger and say, I want to be in this state if you are limiting yourself to only reading from the DOM.

Some frameworks, Ember for example, you have kind of a white box where you can actually inspect the state of the Ember run loop and use that to do some synchronization but it can be very, very hard to coordinate these interactions.

WIL: Yeah. You know, to talk about getting to the solution as a BigTest interactor, which is basically modern page components or page objects. If you ever heard of page objects, it's just a way to encapsulate interacting with big pieces of your pages. It's not a new concept. It's been around for a while but BigTest interactor has kind of a new twist on it where they're immutable, composable interactions that are also convergent, which we'll get into later, which basically means if your buttons not there, it won't click the button until it is there. They're really powerful and they're making really easy and fun to write these tests.

CHARLES: Yeah, they're super powerful. I remember we talked about convergences last time when we talked about BigTest but interactors, I think are definitely a new development. I think we should spend a little bit of time there talking about, not just the power but also the ergonomics of interactors because they are like page components or page objects, except they're scope to the component. Not only do they have all this wonderful stuff where it'll make sure that the component exists before it starts to interact with it and things like that but their composable. If I have a button, then there are certain operations that are valid for that button. I can click it. I can hover over it. I can do all these things. They're the operations that make it unique to the button. Now, those might actually map to real events.

WIL: Similarly, their assertions about that button as well, like as primary is secondary. If this button is repeated throughout your application, you might want to make sure that your form has a primary and secondary button.

CHARLES: Exactly. It really encapsulates all the knowledge of how you can interact with both in terms of taking action and reading state from that button. It almost feels like an accessibility API. It would be easy to write a screen reader if you had these interactors for every single component on the page.

WIL: That's kind of what it is. It's just like you're defining an API around how your user would interact with your application and what your user would expect in the application. That's the point of page objects and interactors as you're defining this user API, essentially.

CHARLES: Yeah and so, really the step that interactor take is that they take the classic page object and it make them composable, so I can have, you kind of touched on this before, a modal dialogue interactor, which is composed out of two button interactors. One for the primary action, one for the secondary action and maybe, it's aware of its own title text, so you can assert on the title text but I didn't actually have to write the individual button interactors for that modal dialog interactor. Then I might have a second modal dialog interactor or a form that's on a modal dialog just composed of the modal dialog interactors and the individual form components, which appear on that particular modal dialog.

WIL: It's essentially how we've been building applications lately with components but this is for page objects in your test if you want to mirror that. You don't have to have one-to-one mappings of an interactor to a component but if you do, it's really powerful.

CHARLES: Yeah. I found that when we have one-to-one interactors, that's when it just feels the best.

WIL: Yeah and on top of this, if you have a component library and your component library exports the interactor that it uses for the component test, like we said, this BigTest technology, they're sprinkled also. We don't have to use interactors in big acceptance tests. We can use them for smaller component tests too, so if we ship these component interactors with the component library, your application that's consuming this component library now can test those components for free, without having to write their own interactors. It can just compose the interactors exported by the library.

CHARLES: Man, I almost want you to repeat that word for word again, just so it can sink in. It's so awesome. Because when you actually go to write your tests, you're not starting from ground zero like, "How do I do this?" They're like, "I'm writing some tests for this thing and I'm using these components and so, I've already got the prepackaged interactions for those components." It's like you start writing your tests. If your tests are a 10-story building, it's like you're starting on Floor 7 and you only have to walk up to Floor 10, instead of slogging up all 10 stories.

WIL: One really helpful interactor that we work within the open source stuff we've been working on is a date-picker interactor because date-pickers can be really complex. Just having that common interactor and have a date-picker on multiple forms where we can just use that one interactor, we don't have to tell every single test how to interact with that date-picker. We just say pick date and pass the date.

CHARLES: Yeah, it's so awesome. That is actually a great example. It doesn't feel scary to write a test for a page that has a date-picker on it or two. If you're doing like a date range or something like that, you're like, "Oh, my God. I don't write the selectors to test this." You just import your date-picker interactor, you set the date, it actually worries about all the low level events and there you go. It feels like you're operating at a much higher level.

WIL: Yeah. The interactor API essentially, you're telling me the test what the user would be doing and what the user would be seeing.

CHARLES: Yeah. It's worth pointing out again. We've identified starting and launching. We've identified set up and tear down but interaction is a core concern of BigTesting, no matter what tool you're using. One of the things that we found as interactors are something that you can sprinkle on literally any test suite if you're testing an interface and it makes it better. We've used it inside big acceptance tests. We use it inside Jest, doing just little component tests. There are people in the BigTest community who have used it to basically, write component tests against a JS DOM and while theoretically, philosophically, you want to make those tests as big as you possibly can, you can use that piece in your test suite.

If you are using a simulated DOM and if you're running a node in a browser, these interactors will still work and you're going to get high fidelity test cases that are resilient to this asynchrony and are composable and if they do have a full-fledged test suite, you can reuse these interactors. They are a really awesome power up that you can bring into your test suite.

WIL: And they are not tied to the framework at all. We use them in React for our stuff but we've also written some in Ember. Robert's written some in the Vue and ported some test and one of the beautiful things we've seen from this is that one interactor goes everywhere. You just write the interactor once and you can use it in Ember, in React, in Vue, in those test suites. If the rest of your test suite is framework agnostic, you have this test suite that you can jump frameworks in your test suites until it works and can test your application with high fidelity.

CHARLES: Yeah, it's fantastic. I remember when we first tried using interactors inside an Ember test suite because Ember comes with like a big kitchen sink in testing set up but interactors just slotted right in and there's absolutely no issue.

WIL: Yeah and there is actually a speed boost even because in most of the Ember test offers a hook into the Ember run loop and interactors are not. There is actually a good speed boos just using interactors.

CHARLES: Yeah. This is a good point. It's a good segue because typically, we think of acceptance tests as being really slow and one of the reasons, even the people [inaudible] acceptance tests or testing big as they think like it's going to take a long time. We found that actually we've been able to maintain a happy medium of testing big but also, having those test be really, really fast. When you say you said a speed boost from using interactors with Ember, where is that speed boost actually come from?

WIL: I mentioned the Ember test offers a hook into the Ember run loop and interactors aren't and the reason of this is because interactors are converging and they wait for things in the DOM to exist before interacting with them. Instead of waiting for the framework to settle, it just waits for the thing to appear and then interacts with it immediately. If you're starting something about a button toward the top of the page, you don't really care that another button at the bottom of the page has rendered yet, unless of course you have assertion about that but if they're converging, you don't need to hook into the wrong loop to wait for the entire page to load, to interact with just one piece of it.

CHARLES: Right. You're just waiting and you say, "I'm expecting something to happen and the moment I detect it, no matter what else is going on, the page could be taking 30 seconds to load but if that button appears and I can interact with it, I can take my action then or I can make my assertion then." It's about kind of removing gates -- artificial gates.

WIL: Yeah. Another common thing that's helped with is animations as most test that are hooked into the run loop, you kind of have to wait for some of these animations to finish before you can even interact with the element and that means if a model has a half second animation where it flies in and you have 30 tests around this modal, those tests are extremely slow now because you have to wait for that modal to come in, whereas --

CHARLES: -- Straight up flaky.

WIL: Yeah, straight up flaky. Whereas in the actual DOM, that modal is inserted pretty immediately and can be interacted with pretty immediately. With interactors, they don't need to wait for the animation to finish. They can just immediately interact with that modal but of course, if you need to wait for the animation to finish, there are options for that as well.

CHARLES: Yeah. If there's some fade in that needs to happen, you can kind of assert on any state and as long as it's achieved at some point, the interactor will recognize it and recognize it at the soonest possible time that it possibly could. I remember getting bitten on one project where the modal animations in particular were so brutal. Not only were they flaky, they just were slow because there was all these manual time outs. It wasn't even a paper cut. It was kind of like a knife cut, like there's someone sitting there and kind of slashing you with a pocket knife. It just was a constant source of pain in your side.

WIL: Yeah and that's how you end up with things like waits and sleeps in your test suite. When you need to wait for the animation to happen or something, you just see a sleep for four seconds with a comment because we have to wait for the components to load in. That's kind of a code now.

CHARLES: Yeah, that's just asking for trouble both in terms of slowness and in terms of it's going to get flaky again. That has been kind of one the most freeing things about working with interactors and working with convergent assertions on which they're based is you just don't ever have to worry about asynchrony. Really, really truly, most of the time, you're writing your tests, like it's all synchronous and that kind of makes sense because from the user's perspective, their consciousness is synchronous and they don't care about the internal run loop. It's just they were making observations in serial and at some point, they're going to observe something, so the interactor sits at that point and really observes the application the way that your user would.

WIL: Yeah. We've mentioned a few times now the convergent assertions, which interactors are based on. A little caveat there if you're using interactors and you're making non-convergent assertions, they might fail or be flaky. That's because interactors wait for the thing to be there to interact with, so as soon as the buttons there, it clicks it but it doesn't wait for after that event has fired and your application has reacted to that event, that's your application is concerned. We need something there like our convergent assertions that can converge on that state and wait for that state to be true before it considers itself passing or in times out.

CHARLES: Maybe we should dig a little bit into convergent assertions. I think the last time we had a public conversation on the podcast about this, this is kind of where we were, like we hadn't built the interactors, we hadn't built these other component pieces of the testing story. We were really focused on the convergent assertion. We've talked a little bit about this but I think it's worth rehashing a little bit because it's a unique way of approaching the system but it's also kind of horrifying when you see how it works under the covers.

I think when we tell people about the fact that it’s basically polling underneath the covers. The timeout is configurable but it's basically polling every 10 milliseconds to observe a state. I remember the first time being confronted with this idea and I was horrified and like my programmer hackles on the back of my neck, like raised up and I was like, "Wait a minute. This is going to be slow. It's going to be computationally intensive."

WIL: Yeah. That was my exact thought too because this is going to be slow. If acceptance tests are slow and we're doing an acceptance test every 10 milliseconds, it's going to be really slow and that's actually not the case completely. It's actually the opposite. They're extremely fast.

CHARLES: It is shockingly fast. You've got to try it to believe how fast it is, how fast you can run acceptance tests.

WIL: Yeah, talking like 100 tests in just tens of seconds.

CHARLES: Right. You basically gated by how fast your framework can render. Your tests are not part of the slowness. Your test --

WIL: And also, memory leaks can be costly too. We experience that recently where we had memory leaks that were slowing down our test but we fixed those up in test and put our backup.

CHARLES: Yeah, because basically, running the assertion or running the convergence is very fast. It's just a very light ping. I kind of think of it is as it is light as the brush of a photon or something that was bouncing off of a surface, so that you can observe it. It's extremely light and most of the time, it's just waiting so the test and the convergence really just gets out of the way. Just because they can run a thousand times or a hundred times in a second, it's doesn't gun it up. But the thing is it means that your tests run as fast as your application will run. You get back to the point... Was it in React where the kind of the key insight is that JavaScript is not the bottleneck? Well, your tests are not the bottleneck.

WIL: Yeah.

CHARLES: I guess this is what it is. I don't know if there's anything else that you want to say about convergences.

WIL: No. We pretty much summed it up there and that's what interactors are based on. That's how they're able to wait for things in a DOM. It basically polls the DOM until it exists and then it moves on and actually does the interaction.

CHARLES: Once again, this is actually a very low level thing on which BigTest is based but this is once again, something that you can use independently. You can write your own convergent assertions. You can write your own convergences that honestly have nothing to do with testing your assertions. It's a free standing library that you can use in your test suite or elsewhere should you choose.

WIL: That doesn't need to be a DOM for BigTest convergence there. I use BigTest convergence in BigTest CLI to converge on the browser being launched. Instead of waiting for the browser to report that, I can just kind of poll and see how that process is doing and the convergence waits for that process to start before moving on.

CHARLES: Right. I guess the best way I've thought about it it's a way to synchronize on observations and not on callbacks. It’s a synchronization mechanism and 99% of the synchronization mechanisms that we're used to, they've involved some sort of callback, a promise, an event-listener, things like that or even a generator where control is handed back explicitly to a piece of code when something happens. Whereas, this is a fundamentally different synchronization primitive, where you are writing synchronous code that's based on observations, so what I observe this, do this. When I observe this, do this. It's extremely robust.

WIL: Yeah, very.

CHARLES: It is a core piece. A fundamental thing that on which interactors are based on, which the CLI is based, I don't know if it's core to writing tests but --

WIL: It definitely helps.

CHARLES: It doesn't helps. We couldn't have BigTest interactor without that.

WIL: No, definitely not.

CHARLES: Because that's what makes it fast, that's what makes it not flaky at all and having those things, I think it makes it easy to maintain because you can work at the interactor level or the level of user interaction and you don't have to worry about synchronization, so the flow of your tests are very natural.

WIL: Yeah. We don't have to explicitly wait for request to be done for making an assertion about your app. That'll just come with convergences, just waiting for test date in application to true.

CHARLES: Let's talk about one more piece of the testing issue because when you're testing big, when you're testing in the browser, there's always the issue of what are you going to do about your API. You got to have your API running. It's just always an issue and this is kind of interesting because this sits at the crossroads of testing big and also, getting the most utility out of your test because in an ideal world, if you're testing really big, you're going to be using a real API. You're not going to poke holes in reality.

WIL: Yeah. One of the things that we avoid in BigTest is poking holes. We're not shallow mounting the components and testing the methods and the results. We're fully mounting these things and fully interacting with them through the full DOM API.

CHARLES: Yeah, exactly, using real browsers. It just occurred to me the irony of us talking about reality being things that are still running inside of a computer processor. I think we've inherited this term from that talk that Justin Searls at AssertJS in 2017. It's a really, really excellent talk. I think he gave it at RubyConf. It's the 'Don’t mock me.'

WIL: Yeah, it's one of my favorite talks.

CHARLES: Yeah, it's a great talk. In it, he talks about the value of a test is a balance of how many holes you poke in reality and sometimes, you encounter a test where all it is like holes in reality. Whether you're mocking this, you're mocking that, you're mocking the DOM, you're mocking the browser, you're mocking your network layer, you're mocking this external API and the more holes you poke, the less useful it's going to be. Network is one of those where it can be very difficult to not poke holes in that reality because it's a huge part of your application. Your frontend application is how it's going to interact with the server but at the same time, servers are gigantic pieces of software themselves, each with their own dependencies, each with their own set up and tear down --

WIL: Have their own concerns.

CHARLES: Yeah, exactly. They might be in a different language. They've got runtime, things like they might need external C libraries and crazy stuff like that. They're their own beast. To get a true big end-to-end test, you going to have to stand up your server but the problem that presents is you want your tests to be also isolatable. If you're a developer, I can go to a repo, I can do an install of my dependencies and I can run the tests without having to do any external dependencies other than the repository and the language in which I'm working.

This is one where we kind of have tried to walk the line of not wanting to poke holes in reality but also, have the test be containable to the actual application. In order to do that, you need something that presents a high fidelity version of the network. You can kind of try and have your cake and eat it too. You want to have something that acts like a server and really acts like a server but it's actually not a server.

WIL: And still poke as few holes as possible and the application and how that's all set up, we don't want to be intercepting methods and responding with fake data. That's not a good way to mock that network.

CHARLES: Right. We want to be calling actual fetches, calling actual XMLHttpRequest. Ideally, if you've got service workers, making actual service worker requests.

WIL: Basically, as far as the application is concerned, it's talking to a real server on us.

CHARLES: Yeah and that's kind of the litmus test for is it a hole in reality or is it just a really great illusion?

WIL: Yeah and that's a good name for Mirage, right? It's a really great illusion.

CHARLES: Yeah. It is a simulation of reality, so we use Mirage, which is something from the Ember testing world but something that we have extracted and made available as BigTest Mirage.

WIL: Yeah. The main difference just being is that we've taken away the Ember dependencies and the run loop stuff. It's just plain JavaScript Mirage. It works exactly the same as you use it in Ember minus the auto imports and the file... Oh, man. I can't think of that word. Aside from automatically importing your files for your server config, you have to do that manually because Ember is what provides that but other than that, it's a form of Mirage. You define models and serializers and factories and all the good stuff.

CHARLES: Right and then you can use those factories and you can use those models to really give a high fidelity server. If you are building something in whatever framework, you can use BigTest Mirage to simulate that network layer. Again, we've used it in a number of different scenarios but having that in place means that you're going to be able to have those high fidelity tests where your application is actually making XMLHttpRequest but it's all isolatable, so that it can be run in repo. This isn't really related to testing but it has a fantastic capability where you can prepopulate, you can use the factories to prepopulate your server with data, so that you can use the application without the actual server being implemented.

WIL: Yeah. That's extremely powerful. That's we were talking about earlier and getting at is the scenarios which are setting up specific, essentially fixtures but you're generating these fixtures. Factories are essentially high level fixtures, network of fixtures.

CHARLES: Yeah, higher order of fixtures.

WIL: Yeah, so the scenarios are just setting up these fixtures for a scenario of your applications, like the backend is down or the list only responds with two items as opposed to 5000 items, something like that. You want to be able to, not only test these things but be able to develop against it and Mirage makes that really easy because you can just start your app with Mirage-enabled point to that scenario and you're there. You have that exhausted scenario to develop in.

CHARLES: If you've never used Mirage, it is really hard to understand just how incredibly powerful it can be. We've used it now on at least four projects, where we did develop the entire first version of the product without any backend whatsoever. It's an incredible product development tool, even apart from testing, that then informs the shape of what the API was going to be. I know we've talked about this on the podcast before but it's really an incredible technology and it is available to you no matter what framework you're using. I think it's one of the best kept secrets in JavaScript development.

WIL: Yeah. That's definitely great. That said, though it does have some fallacies. It's great but it can be a little slow sometimes, so we are eventually working on a BigTest network like another piece of the BigTest pie that you'll be able to sprinkle into your application but in the meantime, praise Mirage.

CHARLES: Yeah. We are going to be offering an alternative or maybe collaborating for another version of Mirage but hopefully, we can make Mirage faster, we will be able to make this thing faster, so that it can use service workers and be used in a bunch of different scenarios.

Just to recap, we've talked about a lot of different components but over the past year, a couple of years, these are the things that we've identified as being really key components as big part of your acceptance testing and really your testing stack. How you're going to start and launch these things? How are you going to set them up and tear them down? How are you going to interact with the application from a user, both in terms of making assertions and how are you going to take action on behalf of the user and still have it be maintainable, have it be resistant to flakiness, have it be performant?

BigTest is the answer to that for those particular areas of the testing story and so, some were using we're using existing components like we use Karma, we use Mirage to date. Those, we did not develop but where we see kind of key pieces of that puzzle missing is where we started writing the BigTest solutions so things like the interactor. Eventually, we are going to make BigTest into a product that's you're going to be able to use kind of out of the box, just like you might install Cypress, where it's a very quick set up and we make all of the decisions about the components for you.

But in the meantime, we're really trying to take our time, identify those pieces of the puzzle and build the software component that fits that piece of the puzzle at the absolute best so when they're polished, use them in a more comprehensive product. Things like convergence, things like interactor, things like BigTest React, BigTest Vue and very soon, BigTest Ember. These are things that you can use today, to make your tests just that much bigger and that much better, especially interactor. It's been an incredible journey this past year as we kind of develop these individual pieces and there's just going to be more goodness to come.

WIL: Absolutely. Right now, I'm working on some validation type API for interactor that I'm hoping to land soon. That'll open up the possibilities of maybe hiding away those convergent assertions a bit more in your tests and just handling this automatically. It'll be pretty good.

CHARLES: It's really exciting. Writing test has got more and more easy and more and more fun over the last year for us. I think we're already starting in a pretty good place. If you have any questions about BigTest, how would folks get in touch with us?

WIL: We have a BigTest Gitter channel. You can find a link to that on the BigTest website: BigTestJS.io. Just ask us questions on Gitter and we'll try to answer them.

CHARLES: And as always, you can ask us directly. You can send email to [email protected] or reach out to us on Twitter at @TheFrontside or you can actually reach out to the BigTestJS Twitter account directly and just call us on Twitter at @BigTestJS. Thank you very much, Wil.

WIL: Thank you, Charles.

Guest:

Dillon Kearns: @dillontkearns | GitHub | Incremental Elm

In this episode, Dillon Kearns joins the show to talk about techniques for experimentation with Elm, making those experiments safe, the concept of mob programming, why you would want to experiment with Elm in the first place, and how you too can begin to experiment with Elm.

Resources:

This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC.

Transcript:

CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode 114. My name is Charles Lowell. I'm a developer here at the Frontside. With me today as co-host on the show is David. Hello, David.

DAVID: Hey, guys.

CHARLES: David is also a developer here at Frontside and we are going to be talking about something that we've been talking, I guess a lot about recently and we're talking about Elm. I think we first started talking about this several years ago and then it kind of simmer down a little bit but recently, it's been top of tongue. With us to talk about Elm today is Dillon Kearns. Welcome Dillon.

DILLON: Thank you so much for having me.

CHARLES: I understand that you are a full time Elm consultant. You have a background as a Lean and Agile coach but have recently transitioned to doing Elm consulting full time. Now, what exactly does that mean in 2018 to be an Elm consultant.

DILLON: Actually a lot of my motivation for getting into Elm consulting in the first place is I kind of realize that Elm to me is just an extension of the things that I was passionate about with Agile and software craftsmanship. I'm trying to help teams have a better experience with their code, make it more maintainable, make it easier to change, make it easier to drive things based on customer feedback and I really believe that Elm helps people do that. I used a lot of the background and experiences that I've had with Agile and Lean coaching and a lot of those same skills, in order to help organizations adopt Elm.

One thing I've seen a lot of teams struggling with is trying out a lot of different frameworks. I've encountered teams that have spent months, very painfully trying out different frontend frameworks and having trouble coming to consensus about that. One of the things that I think really helps address that is having an experimental and iterative approach, that you can really use the scientific method to focus on learning, rather than getting it right the first time. I think that there's really a need to help teams through that process of introducing a new frontend framework like Elm, so that that's why I've gone into full time Elm consulting.

CHARLES: That's an interesting process. It sounds like you really need to be constantly sending out spikes, doing research on whether it's Elm or some other technology to help you kind of bridge the chasm to the next generation. How do you actually do that as an organization? My guess, this kind of a question independent of Elm but maybe we can talk about how you see that play out in the context of Elm.

DILLON: Right and actually, for any listeners interested in that question, I would really highly recommend Grant Maki's ElmConf talk from this year. He spoke about exactly that topic and it was at ElmConf that it's relevant whether your team is considering Elm or looking at other frameworks. I think that the key is you need to get good at experimenting in a way that's low risk and in a way that you can be constantly learning and seeing how these different technologies fit in your codebase and fit for your team.

There's a quote that I really like from Woody Zuill. Have you guys heard of mob programming before?

CHARLES: I heard of mob programming from a paper by Richard Garfield a long, long time ago, almost 20... I don't know if it's the same concept.

DILLON: Yes. It gained a lot of momentum these days. Mob programming is essentially pair programming but with more people involved. I've really enjoyed that process actually. I think it's actually a great way to experiment with different technologies because you get all of the minds together and it's a very good way to kind of transfer knowledge and explore things together but Woody Zuill talks about mob programming and he likes to ask the question, "Why did we begin doing mob programming for the team at Hunter Industries that originally started mob programming?"

People would give answers like, "Because it cuts out code review from the process because you have lots of eyeballs on it in real time," or, "Because it reduces bugs," or, "Because it gives you better quality code. It gives all the best ideas into the product in real time," and all those things are valid points that are really good benefits of mob programming. But he says those things may be true but actually, they're not why we tried mob programming. The reason we tried mob programming was because the team wanted to try it. That's a really important point.

The team needs to be experimenting with things that they're passionate about and they need to be exploring things on their own terms. But with that said, another lesson from that story of kind of his team at Hunter Industries discovering mob programming is that the team didn't discover mob programming in a vacuum. Really, the team discovered mob programming because the team became really excellent at experimenting and evaluating those experiments and then, they like to talk about this phrase that Kent Beck coined, 'turn up the good.' When something is working well, we often focus on the negative things and trying to eliminate those things but what happens if we take the things that are working well and 'turn that dial up to 11.'

CHARLES: Yeah, I love that. I remember in the kind of the original layout of extreme programming, talking about how I really just wanted to turn up all the things that were working for 11 or to 11, so testing, refactoring, incremental releases and things like that.

DILLON: Exactly.

CHARLES: I actually had one question that's maybe a little bit of a diversion. This is actually the first time I've heard of mob programming. It's definitely not the same sort of mob programming I learned about in Richard Garfield's paper. I think it was more referring to massively distributed open source in the form which is really kind of commonplace now that happens on GitHub. I think it's maybe, an obsolete definition of mob programming but how many people would be in a mob like two, three, four, five, six, seven, 10?

DILLON: That's a great question. Really, the answer is of course, it depends. That's a consultant's favorite answer but it really does. My rule of thumb is I find it usually three people is a very nice size for a mob. I find that mobs tend towards around three or four people but that being said, it's important to note that mob programming is all about this idea that what is the true cost of programming.

I think that often we look at programming as the act of writing code, initially and that's a very limited way of looking at coding. Because of course, 90% of our effort is spent maintaining code, making decisions around code, reproducing bugs, fixing bugs, communicating with customers about bugs -- bugs are extremely expensive -- the farther out they get, until eventually they get to the point of a customer discovering them, bugs are in extremely expensive part of software. If we can minimize bugs, that's very valuable.

When you look at programming on this bigger scale and look at the bigger picture of programming, then you realize that you may be able to get one person to write the code faster but then, that person needs to code review it. That person needs to go and ask somebody question down the line when they don't have context because they weren't involved in the decision making.

For example, maybe there's a UX person who doesn't have context on certain choices that were made, so there's a lot of churn, so you can kind of eliminate that churn by getting all the relevant people involved right away and that's the idea. In my experience with mob programming, it works really well to keep kind of a core of around three people. Sometimes, somebody goes up to have a conversation with somebody, take a break or answer somebody's question, maybe somebody from another team has a question that type of thing and so, the team can keep coding as a pair or whatever. But ultimately, the idea is that you get faster because you're building up this shared context and you're not spending as much time down the line answering questions, doing code reviews and things like that.

CHARLES: Right. I see.

DAVID: That kind of matches with my experience. Mob programming on previous teams, the way we had it set up is there was a regular mob programming chat session that the whole team was invited to but it was optional. You can just show up if you wanted to and really, that sort of made it so that there was a set of people who regularly attend -- three to five people in a session -- and they were the core group, essentially.

DILLON: Right. That's another great point. Invitation is a powerful technique. If you're kind of mandating the people try an experiment or work in a certain way, ultimately it's much more powerful to let the team experiment on their own and follow their passion and they'll discover great things. It's about experimenting, rather than choosing specific experiments.

While we're on this topic of kind of the real cost of coding, I think this is a good point to talk about this quality in Elm because, I think that this is one of the things that really motivates me to use Elm myself and introduce it to others is that, I think that Elm really get something about programming where there's a sort of superficial ease of certain techniques that Elm kind of goes beyond and says, "Actually, let's optimize for a different set of things that we think make code more maintainable and more delightful to work with in the long run."

CHARLES: I wanted to also transition between, we were on a little diversions on mob programming but do you use mob programming as explicit technique for introducing Elm when a team is considering adopting it?

DILLON: That's a fantastic question. I absolutely do. Of course, I honor the ways of working in a particular organization or team. I think that's important to do but I do strongly encourage using mobbing as a technique for knowledge sharing and when I'm on-site with a client, I find it extremely powerful as a technique for knowledge sharing and also, let's say you do an experiment, somebody is off in a corner and they're trying out Vue.js or they're trying out Elm or they're trying a particular coding technique. Then they come back to their team and they say, "Hey everybody, I tried this great thing," and now they have to spend this time convincing everybody and saying like, "Wait a minute. You didn't try this, you didn't try it that way. It wouldn't actually work in our context because of this." I think that it's very powerful to have everybody kind of involved in that process so that you can evaluate it together as a team.

CHARLES: Because the thing is like, when you experience win or you experience fail, it's a very visceral feeling and that's the thing that sells you or turns you away. You can argue until you're blue in the face but words have a very limited capacity to convince, especially when compared with like physical and emotional feeling. It sounds like you can get everybody to have that shared experience, whether for the good or for the bad, you're going to arrive at a decision, orders of magnitude more quickly. They have to rely in conviction of that decision spread around the team.

DILLON: Exactly. I think that hits the nail on the head and you say that we have this sort of skepticism of arguments from theoretical conversations, rather than 'show me the money,' but it's actually, try solving a real problem in this and that's exactly as it should be. I think that's one of the big antidote from this problem that I've seen in a lot of environments, where there's this analysis paralysis, especially with the state of the JavaScript ecosystem these days. I think that one of the keys to improving that situation is to get good at trying things, rather than theorizing about things.

We have a tendency to want to theorize and when we do that, then we say, "Can it solve this problem? Can it solve this problem? Can it solve that problem?" You can talk about that until the cows come home but it doesn't get you anywhere and it doesn't really convince anybody of anything. The key is to find very small experiments and what I really recommend and what I'm dead focused on when I'm initially working with a client is getting something into production.

Now, that doesn't mean that you need to have a road map for turning your entire application into Elm. In fact that's the whole point, is that you're not trying to do that. The point is you're trying to get as realistic of an experience as possible for what problems might occur if we do this? Will the team enjoy working with this language? Will it work well with our built pipeline? Will there be any unforeseen issues? You don't know until you actually try it, so you've got to try it and you've got to try it in tiny, tiny steps and low risk experiments.

CHARLES: Right but you've got to try it for real. You don't want to try it with a TodoMVC.

DILLON: Exactly. It needs to be meaningful, to really have a good understanding of what it's going to be like.

CHARLES: I would say that I tend to agree but I've definitely encountered the counterargument and I also think this counterargument makes sense or perhaps where the pushback lies is if I'm constantly experimenting, then what I'm doing is I'm internally fragmenting my ecosystem and there is power in similarity. Any time you introduce something different, any time you introduce one fragment, you're introducing complexity -- a mental complexity -- like maybe I have to maintain my Elm app and I also have to have my Legacy... Or not Legacy, I've got my other JavaScript tool kit that does it in one way. Maybe I've got a couple of more because I've run these other experiments. I'm not saying that there is one way but there is power in uniformity. There is power in diversity. Where do you find the balance?

DILLON: Those are all excellent points. To me, I think really the key is it's about the scientific method, you could say. The thing with the scientific method is that we often forget the last part. We get really good at hypothesizing about things. Sometimes people leave it at that, which we kind of just discussed. Sometimes, people go past the hypothesizing stage and they actually run the experiment and that's great. But then, the majority of people, if they get to that point, will forget to do the last step which is to evaluate the results.

I think the key here is you need to be experimenting and this is what it means for it to be a low risk experiment. It means that you're not setting yourself off in a direction where you can't turn back. You want to set it up in such a way that you can turn away from it with minimal cost. One of the things that is really helpful for that is if you build a tiny, independent, little widget in your application, try building that in Elm. Some people will do that with a little sort of login badge in the corner of their application.

One of the teams where I've introduced Elm at a Fortune 10 company, actually where we introduced Elm, we started out with just a tiny little table in one page and if we wanted to back that out, it would have been trivially easy but we decided that we wanted to go in further and invest more.

CHARLES: That makes a lot of sense. Effectively, you need to have a Plan B. Don't sync all of the available time that you have to invest in an experiment. Make sure that you have a Plan B and if you need to do this widget or this table in Angular or React or Ember or whatever, you are thinking about that -- how would that work.

DILLON: Exactly and the thing with experiments is the purpose of an experiment is not to build something. It's to learn. I really like this kind of ethos of lean startup, which I think is really getting much more into the mainstream in the software industry, which is a wonderful thing. The idea of lean startup, the kind of core concept is this idea of validated learning. Basically, in an environment where there's uncertainty, which is pretty much most of the things you're doing in software, the main goal is you're not shipping a product like you would be if you're trying to manufacture cars as quickly as possible. The main thing that you're producing is what they call 'validated learning' and so, you want to minimize the amount of time it takes to validate or invalidate your assumptions about something and then, you want to make it as cheap as possible to move on from that.

CHARLES: I like that. So if you're going to organize your development process around this principle or maybe not organize it but integrate it into development process, how do you know that you're conducting a healthy number of experiments, versus I may be conducting too many experiments? Is there a metric that you can look at? We need to have this many experiments running at all times or this is just too many or something else.

DILLON: That's a really interesting question. I think I would tend to think about that more again, as looking at the way the experiments are run, rather than 'are there too many experiments?' That's just not a problem that I've seen there being too many experiments. The pain that we tend to really see in environments where experiments are hurting teams is the way the experiments are being done. It's hard to backtrack from those experiments and as you were saying before, you kind of put yourself down this path where you can't walk it back and you create this sort of rift in the way the code is being written, which makes it more difficult to work in that codebase.

The thing with experiments is they can have really big payoff. Now, you want to make sure that you're not just going in and picking up every shiny object that you see. One thing that can keep you honest with that is if you're kind of coming up with a hypothesis before you start. If you're saying, "This is the value to our business and to our team if we attempt this thing and this is what will prove that it seems to have that value and this is what will tell us, 'Actually, it doesn't have that value and we should drop it and cut our losses.'"

CHARLES: That's a great heuristic. As you're saying and imagining how that might have saved my bacon in the past because I've definitely made the mistake of playing with too many shiny objects and picking things because I didn't fully evaluate what I thought the value. I was explicit with myself about what is the value that is going to bring to this project or this business. I have a theory about it but I am not thinking what is my hypothesis and how am I going to validate or invalidate? I'm thinking, I've got a short term pain that I'm experiencing and I'm grasping for this thing, which I think will solve it and I'm not properly evaluating how it's going to affect me long term.

DILLON: Right and that could be a great team practice to play around with is often, teams will kind of come up with action items out of retrospectives. One thing that I think can be really beneficial for teams is to kind of flip that notion of doing action items which again, it's really just doing the middle part of the experiment where you're conducting the test but you cut out the hypothesis part and the evaluation part. Try to bring that into your team's retrospective and try to have explicit hypotheses in the retrospectives and then, in the next retrospective, evaluate the results.

CHARLES: All right. I will definitely keep that in mind but this feels like a fresh take on kind of how you manage software development that I haven't encountered too much, being more scientific about it. It sounds like science-oriented development.

DILLON: Right.

DAVID: I like that.

DILLON: There are a lot of buzzwords these days in software development, in general and it's really becoming a problem, I think in the Agile community but really, what it boils down to is these basic elements and basing decisions on feedback is one of those fundamental unit. You can call the scientific method, you can call it lean startup and validated learning, you can call it agile, you can call it whatever you want but ultimately, you need to be basing things on feedback.

I think of it almost like our nerves. There's actually a disorder that some people have, which can be fatal, which is that their nerves don't tell them when they're feeling pain. I think this is a great analogy for software because that can happen to companies too. They don't feel the pain of certain decisions not landing well. Because they're not getting feedback from users, they're not getting feedback from metrics and recording, they're not getting feedback from doing that final evaluation step of their experiments, so when you fall on the ground, a small cut could be extremely harmful because you don't know the damage it's doing to you.

CHARLES: I think that is a good analogy. One of the things that I'm curious about is we've been discussing a lot of techniques for experimentation and how you can integrate that into your process and how you can make your experiments safe, so let's talk a little bit about -- first of all, two things -- why would I want to experiment with Elm in the first place? Because ultimately, that's why we're here and why we're having this conversation. What's compelling about it that would make me want to experiment? And then how can I begin to experiment with Elm?

DILLON: I actually just published a blog post yesterday. It's called 'How Elm Code Tends Towards Simplicity.' To your question of why would a team consider Elm, I kind of talk a little bit in this blog post about a case study at a Fortune 10 company where I introduced Elm to a few of the teams there. One of the teams there, we had actually seen an Angular project that they had worked on and often, in an enterprise environment, you have projects moving from one team to another.

I actually had my hands on this Angular project. It kind of moved over to another team and we were experiencing some major pain trying to make changes in this codebase. Even making the simplest change, we were finding that there were a lot of bugs that would be introduced because there's some global variable. There's some implicit state. Sometimes, it was even reaching in and tweaking the DOM and really, the topic of conversation at our team lunches was how afraid we were to touch this codebase.

Fast forward a few months and this team was asking my advice on picking a new frontend framework and I introduced them to Elm. They took a run with it and it was pretty remarkable to see this same team that had really struggled with AngularJS and they didn't really have a strong sense of what were the best practices. They weren't getting any guidance from the framework itself and the tooling around it and they actually loved the experience of working with Elm because they were saying, "This is amazing. Maybe it takes a little time to figure out how to solve a particular problem on Elm but once we do, we know that we've done it in a solid way."

One of the things that I think is most powerful about Elm is that it keeps you from shooting yourself in the foot. I think that's a really good headline kind of summary of what I love about Elm. For example, tweaking the DOM. Now, it might seem like a pretty obvious thing that we just won't tweak the DOM and that's fair enough. That might not be a problem for a lot of teams. People wouldn't even reach for that technique because they're disciplined about it. But at a certain point, you start taking on enough things and then go from kind of those basic things that are going to make your code more unreliable and unsafe like tweaking the DOM and you start getting into the realm of best practices.

There's so much discussion these days in the JavaScript community about best practices, which is great. It's great to discuss that but my concern is that there's a new best practice each week and the team has to agree on it, you have to find techniques for enforcing it, people have to make sure that these best practices are being followed in code reviews. Then when you look at a given piece of code, you have to trust that those best practices are being followed, so it requires a lot of work to make sure, in your reducers, in redux that you are not mutating anything. With Elm, data is just immutable. That's just how it is.

There are a lot of these kind of things that are baked into the language and the expressivity of the type system allows you to bake in your own constraints. One of the things that I find really compelling about Elm is its design really prevents you from shooting yourself in the foot and it gives you tools for making sure that you take it even a step further and it helps you enforce these best practices at a compiler level.

CHARLES: Now what's interesting here is it's almost like the opposite tension of experimentation is a work, right? like here, we have an example of uniformity being the more powerful track but then inside the actual macroscopic process, you want a lot of experimentation and diversity. But at the microscopic level, inside your application, it sounds like you want less experimentation and you derive a lot of strength from that but --

DILLON: That's a great point.

CHARLES: -- Experiments that are possible, yeah.

DILLON: I think that there is a lot of pain these days in the JavaScript community. We hear people talking often about JavaScript fatigue and it's a real thing. It takes a lot of work to stay on top of the latest best practices and frameworks and that can be a lot of fun. I love learning about the latest new frameworks and tooling but ultimately as you're saying, we don't want that experimentation so much about the fundamentals. We want some dependable, solid fundamentals and then we want the experimentation to happen within there. I think that's exactly what we see in the Elm ecosystem.

We have a single kind of data store or way of managing state in Elm. It's called the Elm Architecture. In fact, it's what Redux is based on and it worked extremely well and you don't have to experiment with different data stores in Elm because that's just what Elm code looks like. Now, if you want to experiment in Elm, then there is a lot of innovation happening. One of my favorite things about Elm is that the compiler and its expressiveness has sparked a lot of creativity. One of my favorite things about Elm is the library called Elm UI.

Actually, a client that I'm working with right now, it's a really interesting case study. They are kind of a very small startup. They just kind of branched off of a larger startup. They're building some tooling for this ecosystem. They were engineers at a company called Procore that does cloud document management for construction companies. They wanted to get a product-ready for a big conference for their potential clients. The reason they brought me in to help them was because they wanted to reach this ambitious target of being able to do a demo of this brand new product at this conference and they wanted to iterate very quickly. One of the things that really drew them into Elm in the first place is this library Elm UI.

Elm UI essentially, Richard Feldman gives a talk on it, where he uses the analogy of it being treating CSS and HTML as bytecode for your views. I think that's a really apt way to put it. If you break down this idea of CSS -- Cascading Style Sheets, it removes the cascading part of CSS and it removes the sheets part of CSS. What you're left with is a way of expressing style and it's a way of expressing style that is able to part ways with all of the baggage of the entire history of backwards compatible decisions that CSS has ever made.

If you want to vertically aligned something, then you just say, "Align vertically," you know, center vertically. If you want to center something horizontally, you say center X. It creates a high level language for expressing views. My experience with Elm UI, this may not be the right choice for every team but I love it. I use it on all of the projects that I maintain personally. I love using it because it gives you that same sense of invincibility refactoring that you get with Elm, which is remarkable that you could have that feeling with managing views.

CHARLES: It's definitely something that feels like a dark art and it can't be called science. It's an art. It's a science for some people but it's historically been a dark art and something fiddly to work with. In terms of being able to make the experiment with Elm, when we talked a little bit about why you might want to experiment with it in the first place, what the business case is, I guess my next question is or a question that immediately comes to mind is supposing that we have decided to experiment with this, how do you mitigate that experiment?

We talked about lowering the cost, having a way to turn away from it, having a way to make it inexpensive. For example, one of the things that I think of when evaluating a new technology is how well can I use it with old technologies. I have a lot about best practices in my tool bag. We all do. We got our all favorite libraries and pathways that are just familiar to us. One of the things that I've noticed is when adopting a new technology, one of the things that makes it easy to experiment with is how well it works with the existing technologies. I know that, we talked about Elm UI, kind of rethinking style in CSS and your views and Elm itself as a completely different language within JavaScript, that can be both liberating but it can also be limiting in the sense that I can't reach back for my existing tool if no tool exists in this new space.

The kind of experience that I've had where this is really worked is systems like JRuby or Clojure, where there's a very clear pathway to be able to use Java libraries from those environments, so you always have kind of an escape hatch. What's that like in Elm?

DILLON: This is a really interesting conversation because it highlights, in some ways some of the most defining features of Elm. In terms of how do you kind of pull Elm into an existing application, there are a lot of different techniques for that. It's pretty straightforward to create a little Elm app. We usually don't call them components for reasons that we can get into if we want to but that's a whole can of worms.

But if you've got a little Elm application that you want to use to render a widget on your page, then it's as simple as just calling Elm.yourmodule.init and rendering it onto the page there. That's quite straightforward and if you want to interface with your existing code there are several ways to do that. There's something called port in Elm, which is how you kind of communicate by sending these messages and data back and forth between your Elm app and JavaScript.

Now, this is one of the decisions, I think that defines Elm as the language and the reason this is important is because Elm decided not to make the choice that a lot of other compile to JS languages do. For example, if you look at ReasonML or PureScript or a more extreme example, TypeScript. TypeScript is a superset of JavaScript, so it's trying to allow you to gradually introduce this to get some incremental improvements for your JavaScript code, so it's extremely easy to experiment with it, which we've talked about the importance of experimentation.

Now, the challenge with this technique, the tradeoff here is it's great, that it then becomes very easy to transition into it and that's an excellent strategy for the goals of TypeScript. Elm has a different set of goals, so the things that elm is focused on giving you is a truly type-safe experience. When you're working with Elm, if your Elm code says that this data is a float, then it is a float. Either, it is a float or that code is not being run and so, that's very different than the experience in TypeScript where you have these escape hatches.

This is an inevitable choice for any compiled to JS language. Are you going to have escape hatches or not? Elm is really the only language out there, I think that chooses not to have escape hatches and that is actually the thing that that I love about the language because that's the only way you can truly have guarantees, rather than, "Yeah, I'm pretty sure that these type guarantees hold."

DAVID: Yeah, wishes and dreams.

DILLON: Yeah.

CHARLES: What does it mean to have no escape hatches? because you talked about ports. Does it mean like it's impossible to use an external JavaScript library?

DILLON: That's an excellent question. You absolutely can use JavaScript libraries. It means that it's being explicit and upfront about the fact that there's uncertainty in these areas. That's what it comes down to. Take for example dealing with JSON. In a JavaScript application, what we get when we're dealing with JSON is you make a request up to the server, you have some callback that passes in the data you get back and then you start pulling bits and pieces off of it and you say 'response.users subzero.firstname' and you hope that none of those things are null, none of those types are different than what you expected.

In a way, it's kind of letting you pretend that you have certainty there when in fact, you don't and with Elm, the approach is quite different. You have to explicitly say, "I expect my response to have this shape. I expect it to be a list of things, which have a first name and last name which are strings," and then Elm says, "Okay, great. I'm going to check your assumptions," and if you're right, then here you go and you're in a well typed-space where you know exactly what the types are and if you're wrong, then that's just another type of data, so it's just a case statement where you say, "If my assumptions were correct, then do something and if my assumptions were incorrect, then you decide what to do from there."

CHARLES: Right. For me, it sounds like there is some way because ultimately, I'm going to be getting unstructured but I'm going to be getting JSON back from the server and maybe, I have some library that's going to be doing that for me and enhancing it and adding value to that JSON in some way. But then at some point, I can present it to Elm but what you just saying is I need to be complete in making sure that I handle each case. I need to do or handle the case. Explicit about saying if the assumptions that Elm wants to make, turn out to not be true, Elm is going to make me handle the case where those assumptions were not true.

DILLON: Exactly. I think that TypeScript of any type is the perfect illustration of the difference. TypeScript of any type is sort of allowing you to say, "Don't type check this. Trust me here," and Elm's approach is more kind of just be explicit about what you want me to do if your assumptions are incorrect. It doesn't let you kind of come in and say, "No, I know I'm going to be right here."

CHARLES: Right but there is a way to pass data structures back and forth.

DILLON: There absolutely is and actually, there's a technique that's starting to gain some traction now, which I'm really excited about, which is rather than using this sort of JavaScript interrupt technique we talked about, which is again, it's very much like communicating with a server where you're kind of sending messages and getting data back -- getting these messages with data back. But there's an alternative to that which is using web components. Actually, there's quite good support for assuming that you don't need to be compatible with Internet Explorer.

Basically in a nutshell, if you can wrap a sort of declarative web component around anything, it could be a Google Maps API, it could be a syntax highlighting JavaScript library, something that you don't have an Elm library for but you want to use this JavaScript library, it's actually quite a nice experience. You just render that custom element using your Elm code just as you would any other HTML in Elm.

CHARLES: Yeah I like that, so the HTML becomes the canvas or composition with other JavaScript and the semantics are very well-defined and that interface is actually pretty thin.

DILLON: Exactly and the key again is that you wanted to find a declarative interface, rather than an imperative one where you're kind of just doing a series of statements where you say, "Do this and then set this value and then call this and then set this call back." Instead, you're saying, "Render this Google Maps custom element," which is centered around these coordinates and has this zoom level on. You declaratively give it the bit of information that it needs to render a particular view.

CHARLES: Okay. Then I guess the final question that I have around this area is about being able to integrate existing tools and functions inside of an Elm application. Because it sounds like you could theoretically develop large parts of your application, is there a way that you can actually have other areas of your application that are not currently invested in Elm still benefit from it, in the sense for kind of need of JavaScript APIs that Elm can make available.

DILLON: Right, so you're kind of talking about the reverse of that Elm reaching out to JavaScript. You're asking about, can JavaScript reach out to Elm and benefit from some of its ecosystem?

CHARLES: Exactly. I say that is that another potential vector for experimentation.

DILLON: It's a really interesting thought. I haven't given it too much thought, to be honest but I actually have heard it come up before and my gut feeling is that it's probably more fruitful to explore the inverse, reaching out to JavaScript from Elm and the reason is kind of the main appeal of Elm is that when you're operating within Elm, you have this sense that if it compiles, it works. Because again, this central decision to not allow escape hatches is what allows you to have that sort of robustness, so you have this feeling of bullet proof refactorings and adding new features seamlessly where you change your data modeling to say, "Here's this other case that can be represented," and then suddenly, the Elm compiler says, "Tell me what to do here, tell me what to do here and tell me what to do there," and you do it and your app is working. That's the real appeal of Elm, I think and you don't really get much of that by just calling out to an Elm library from within JavaScript. That's my gut feeling on it.

CHARLES: Okay, that's fair enough. On the subject of interrupt and using tools like JSON, you actually maintain a GraphQL library for Elm. You probably have a lot of experience on this. Maybe we can talk about that as a concrete case that highlights the examples.

DILLON: Yeah. I think to me this is one of the things that really highlights the power of Elm, to give you a really amazing refactoring and kind of feature creating experience. A lot of Elm libraries are prefaced by the author name, so it's still DillonKearns/ElmGraphQL. I spoke about it this year at ElmConf.

In a nutshell, what it does is it actually generates code based on your GraphQL schema. For anyone who doesn't know, GraphQL is just kind of a language for expressing the shape of your API and what types of data can return. What DillonKearns on GraphQL does is it looks at your GraphQL schema and it generates an API that allows you to query that API. using this library, you can actually guarantee that you're making a valid query to your server. Again, you get this bulletproof experience of refactoring in Elm where you can do something like make a change in your API and recompile your Elm code and see whether you've made a backwards incompatible change.

All of this effort of doing sort of this JSON decoding I was talking about earlier where you kind of have to explicitly say, "These are my assumptions about the shape of the JSON that I'm getting back." When you're using this library, you no longer need to make any assumptions because you're able to rely on this sort of schema of your API and so you know when you're requesting this data, you don't have to run it, see if it works and then tweak it and run it again -- this sort of cycle of checking your assumptions at runtime.

It moves those assumptions that you're making from runtime to compile time and it can tell you when you compile your application, it can say, "Actually, this data you're requesting, it doesn't exist," or, "It's actually called this," or, "This is actually the type of the data."

CHARLES: Right. I love that. How do you do that? Because it seems like you've got a little bit of a chicken and the egg problem because the schema is defined outside of Elm, so you have to be able to parse and understand the schema in order to generate the Elm types to be able to compile Elm code against them. Maybe I'm not --

DILLON: That's exactly right. That's exactly what it does. Now, the nice thing is that GraphQL is really designed for these types of use cases. It supports them in a first class way. If you have a GraphQL API, that means you have built into it whether you know about it or not, a way to introspect the schema. All of the queries for kind of interrogating that GraphQL server and asking what types of data does this return, what are all your queries that I can run, it's built into it by the framework, so that comes for free. Getting up and running with this package I built is as simple as running a little npm CLI, pointing it to either your URL for your server or the JSON form of your schema, if you prefer and then, it generates the code for you.

CHARLES: Wow, that sounds fantastic. This is the exact kind of thing that feels like it would be cool if I could just start using this library to manage the GraphQL of my application but I'm consuming that GraphQL from other JavaScript but it's the Elm code that's managing it. Do you see what I mean?

DILLON: I hadn't considered that. I guess you could. You're right. Maybe I’m so smitten with Elm that it's hard to see an in-between but I guess, you could get some benefits from that approach.

CHARLES: Right and as an experiment of course.

DILLON: There you go. There you go.

CHARLES: All right. With that, I think we'll wrap it up. Thank you so much, Dillon for coming on and talking with us on the podcast.

DILLON: My pleasure. I really enjoyed the conversation.

CHARLES: I actually got so many great tidbits from so many different areas of software development in Elm but also, just in kind of other things that I'm interested in trying. It was a really great conversation.

DILLON: I had a lot of fun and I love discussing these things. For any listeners who are interested in this stuff, feel free to reach out to me on the Elm Slack or on Twitter. I'm at @DillonTKearns. I'm also offering a free intro Elm talk for any companies that are kind of entertaining the idea of doing an experiment with Elm. If you go to IncrementalElm.com/Intro, you can find out about some of the talks that I'm offering.

CHARLES: All right. Well, thank you very much and we, as always are the Frontside. We build software that you can stake your future on and you can get in touch with us at @TheFrontside on Twitter or [email protected] on email. Please send us any questions you might have, any topics that you'd like to hear about and we look forward to hearing from you and we will see you next week.

Guest:

Philip Poots: GitHub | ClubCollect

Previous Episode: 056: Ember vs. Elm: The Showdown with Philip Poots

In this episode, Philip Poots joins the show again to talk about the beauty of simplicity, the simplicity and similarities between Elm and Ruby programming languages, whether Elixir is a distant cousin of the two, the complexity of Ember and JavaScript ecosystems (Ember helps, but is fighting a losing battle), static vs. dynamic, the ease of Rails (productivity), and the promise of Ember (productivity, convention).

The panel also talks about the definition of "quality", making code long-term maintainable, and determining what is good vs. what is bad for your codebase.

Resources:

This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC.

Transcript:

CHARLES:: Hello, everybody and welcome to The Frontside Podcast, Episode 113. My name is Charles Lowell. I'm a developer here at the Frontside and with me today are Taras Mankovski and David Keathley. Hello?

DAVID:: Hey, guys.

TARAS: Hello, hello.

CHARLES:: And we're going to be talking with a serial guest on our serial podcast, Mr Philip Poots, who is the VP of engineering at ClubCollect. Welcome, Philip.

PHILIP: Hey, guys. Thanks for having me on.

CHARLES:: Yeah. I'm actually excited to have you on. We've had you on a couple of times before. We've been trying to get you on the podcast, I think for about a year, to talk about I think what has kind of a unique story in programming these days. The prevailing narrative is that folks start off with some language that's dynamically typed and object oriented and then at some point, they discover functional programming and then at some point, they discover static programming and they march off into a promised land of Nirvana and no bugs ever, ever happening again. It seems like it's pretty much a straight line from that point to the next point and passing through those way stations.

When I talk to you, I guess... Gosh, I think you were the first person that really introduced me to Elm back at Wicked Good Ember in 2016 and it seemed like you were kind of following that arc but actually, that was a bit deceptive because then the next time I talked to you, you were saying, "No, man. I'm really into Ruby and kind of diving in and trying to get into Ruby again," and I was kind of like, "Record scratch." You're kind of jumping around the points. You're not following the preordained story arc. What is going on here? I just kind of wanted to have a conversation about that and find out what the deal was and then, what's kind have guided your journey.

PHILIP: There was one event and that was ElmConf Europe, which was a fantastic conference. Really, one of the best conferences I've been to, just because I guess with the nature of early language, small conference environment. There's just a lot of things happening. There's a lot of people. Evan was there, Richard Feldman was there, the leading lights of the Elm community were there and it was fantastic.

But I guess, one thing that people have always said to me is the whole way track is the best track of the conference and it's not something I really appreciated before and during the breaks, I ended up talking to a guy called Michel Martens. He is the finder of a Redis sourcing company and I guess, this was just a revelation to me. He was interested in Elm. He was friends with the guys that organized the conference and we got talking and he was like, "I do this in Ruby. I do this in Ruby. I did this in Ruby," and I was like, "What?" and he was like, "Yeah, yeah, yeah."

He's a really, really humble guy but as soon as I got home, I checked him out. His GitHub is 'soveran' and it turns out he's written... I don't know, how many gems in Ruby, all with really well-chosen names, very short, very clear, very detailed. The best thing about his libraries is you can print them out on paper. What I mean by that is they were tiny. They were so small and I guess, I just never seen that before. I go into Ruby on Rails -- that was my first exposure to programming, that was my first exposure to everything -- unlike with Rails, often when you hit problems, you'd start to dive a bit deeper and ultimately, you dive so deep that you sunk essentially and you just accepted, "Okay, I'm not going to bend the framework that way this time. Let's figure out how everyone else goes with the framework and do that."

Then with Ember when I moved into frontend, that was a similar thing. There were so many layers of complexity that I never felt like had a real handle on it. I kind of just thought this was the way things were. I thought it's always going to be complex. That's just the nature of the problem. That's just the problem they're trying to solve. It's a complex problem and therefore, that complexity is necessary.

But it was Elm that taught me, I think that choosing the right primitives and thinking very carefully about the problem can actually give you something that's quite simple but incredibly powerful. Not only something quite simple but something so simple that it can fit inside your head, like this concept of a program fitting inside your head and Rails, I don't know how many heads I need to fit Rails in or Ember for that matter and believe me, I tried it but with Elm, there was that simplicity.

When I came across this Ruby, a language I was very familiar with but this Ruby that I had never seen before, a clear example was a templating library and he calls it 'mote' and it's including comments. It's under a hundred lines of code and it does everything you would need to. Sure, there were one or two edge cases that it doesn't cover but it's like, "Let's use the trade off." It almost feels like [inaudible] because he was always a big believer in "You ain't going to need it. Let's go for that 80% win with 20% effort," and this was like that taken to the extreme.

CHARLES:: I'm just curious, just to kind of put a fine point on it, it sounds like there might be more in common, like a deeper camaraderie between this style of Ruby and the style encouraged by Elm, even though that on the surface, one is a dynamically typed object oriented language and the other is a statically typed functional language and yet, there's a deeper philosophical alignment that seems like it's invisible to 99% of the discussion that happens around these languages.

PHILIP: Yeah, I think so. I think the categories we and this is something Richard Feldman talks. He's a member of the Elm community. He does a lot of talks and has a course also in Frontend Masters, which I highly recommend. But he often talks about the frame of the conversation is wrong because you have good statically typed languages and you have bad statically typed languages. You have good dynamic languages and you have bad dynamic languages. For all interpretations of good and bad, right? I don't want to start any wars here.

I think one of the things that Elm and Ruby have in common is the creator. Matz designed Ruby because he wanted programming to be a joy, you know? And Evan created Elm because he wanted programming to be a delight. I think if you experience both of those, like developing in both of those languages, you gain a certain appreciation for what that means. It is almost undefinable, indistinguishable, although you can see the effects of it everywhere. In Ruby, everything is an object, including nil. In Elm, it's almost he's taken everything away. Evan's taken everything away that could potentially cause you to stumble.

There's a lot to learn with Elm in terms of getting your head around functional mindset and also, working with types but as far as that goes, people often call it like the Haskell Light, which I think those are a disservice to Elm because it's got different goals.

CHARLES:: Yeah, you can tell that. You know, my explorations with Elm, the personality of Elm is 100% different than the personality of Haskell, if that is even a programming term that you can apply. For example, the compiler has an identity. It always talks to you in the first person, "I saw that you did this, perhaps you meant this. You should look here or I couldn't understand what you were trying to tell me." Literally that's how the Elm compiler talks to you. It actually talks to you like a person and so, it's very... Sorry, go ahead.

PHILIP: No, no, I think the corollary to that is the principle of the surprise in Ruby. You know, is there going to be a method that does this? You type it out and you're like, "Oh, yes it is," which is why things like inject and reduce are both methods in enumerable. You didn't choose one over the other. It was just like, "Let's make it easy for the person who's programming to use what they know best."

I think as well, maybe people don't think about this as deeply but the level of thought that Evan has put into designing Elm is crazy, like he's thought this through. I'm not sure if I said this the last time but I went to a workshop in the early days in London, which is my kind of first real exposure to Elm and Evan was giving the workshop. Someone asked him, "Why didn't you do this?" and he was like, "Well, that might be okay for now but I'm not sure that would make so much sense in 10 years," and I was kind of like, "What?" Because JavaScript and that ecosystem is something which is changing like practically hourly and this is a guy that's thinking 10 years into the future.

TARAS: You might have answered it already but I'm curious of what you think is the difference, maybe it just comes down to that long term thinking but we see this in JavaScript world a lot, which is this kind of almost indifference to APIs. It almost doesn't really matter what the API is for whatever reason, there seems to be a big correlation between the API that's exposed with the popularity of the tool. I think there are some patterns, like something that's really simple, like jQuery and React have become popular because of the simplicity of their APIs. What the flip side to that? What other ways can APIs be created that we see in JavaScript world. Because we're talking about this beautiful APIs and I can relate to some of the work that Charles has been doing and I've been doing microstates but I wonder like what would be just a brief alternative to that API, so it's kind of a beautiful API.

PHILIP: I don't know if anyone is familiar with the series of essays 'Worse is Better' like East Coast versus West Coast, from Richard Gabriel. The problem is, I guess and maybe this is just my understanding over my paraphrase of it, I'm not too familiar with it but I think that good APIs take time and people don't have time. If someone launches a V1 at first and it kind of does the job, people will use that over nothing and then whenever they're happy with that, they'll continue to use it and develop it and ultimately, if she's market share and then that's just the thing everyone uses and the other guy's kind of left behind like, "This is so much better."

I guess this is a question, I think it was after Wicked Good Ember, I happened to be on the same trend as Tom Dale on the way back to New York and we started talking about this. I think that's his big question. I think it's also a question that still has to be answered, which is, "Will Elm ever be mainstream? Will it be the most popular thing?" aside from the question of whether it has to be or not. For me, a good APIs good design comes from understanding the problem fully --

CHARLES:: And you can understand the problem fully without time.

PHILIP: Exactly and often, what happens -- at least this is what happens in my experience with the production software that I've written -- is that you don't actually understand the problem until you've developed a solution for it. Then when you've developed a solution for it, often the pressures or the commercial pressures or an open source is [inaudible] the pressures of backwards compatibility, mean that you can never refactor your way to what you think the best solution is and often, you start from scratch and the reality is people are too far away with the stuff you wrote in the past about the thing you're writing now. Those are always kind of at odds.

I think there are a lot of people that are annoyed with Elm because the updates are too slow, it relies on Evan and we want to have a pool request accepted. All of the things that they don't necessarily recognize like the absence of which make Elm an Elm, if you know what I mean. The very fact that Evan does set such a high standard and does want everything to go through his personal filters because otherwise, you wouldn't gain the benefits that Elm gives you. The attention is very real in terms of I want to shift my software now and it becomes easier then.

I think to go to a language like JavaScript, which has all of the escape hatches that you need, to be able to chop and change, to edit, to do what you need to do to get the job done and let's be quite honest, I think, also with Elm, that's the challenge for someone who's not an expert level like me. Once you hit a roadblock, you'll say, "Where do I go from here?" I know if I was using JavaScript, I could just like hack it and then clients are happy and everything's fine and you know there's a bit of stuff in your code that you would rather wasn't but at the end of the day, you go home and the job's done.

DAVID:: Have you had to teach Elm to other people? You and I did some work like I've seen you pair with someone and guide them through the work that they needs to get done. If you had a chance to do something like that with Elm and see how that actually happens, like how do developer's mind develops as they're working through in using the tool?

PHILIP: Unfortunately not. I would actually love to go through that experience. I hope none of my developers are listening to this podcast but secretly, I want to push them in the direction of Elm on the frontend. But no, but I can at least make from my own perspective. I find it very challenging at first because for me, being a Ruby developer and also, I would never say that I understood JavaScript as much as I would have liked. Coming from dynamic language, no functional experience to functional language with types, it's almost like learning a couple of different things at the same time and that was challenging.

I think if I were to take someone through it, I would maybe start with a functional aspects and then move on to the type aspects or vice versa, like try and clearly breakdown and it's difficult because those two are so intertwined at some level.

Gary Bernhardt of Destroy All Software Screencast, I watch quite a bit of his stuff and I had sent him an email to ask him some questions about one of the episodes that he did and he told me that he done the programming languages course, I think it's on Coursera from Daniel Grossman, so [inaudible] ML which is kind of the father of all MLs like Haskell and also Elm. I find that really helpful because he broke it down on a very basic level and his goal wasn't to teach you ML. It was to teach you functional programming. It would be a very interesting exercise, I think.

I think the benefit that Elm gave you is you get to experience that delight very quickly with, "Oh, it's broken. Here's a nice message. I fix the message. It compiles. Wow, it works," and then there's a very big jump whenever you start talking about the effects. Whenever you want to actually do something like HTTP calls or dealing with the time or I guess, the impure stuff you would call in the Haskell-land and that was also kind of a bit weird.

CHARLES:: Also, there's been some churn around that, right?

PHILIP: That's right. When I started learning, they had signals, then they kind of pushed that all behind the scenes and made it a lot more straightforward. Then I just mastered it and I was like, "Yes, I know it," and then I was like, "All right. I don't need to know it anymore."

This is the interesting thing for me because at work, most of our work now is in Elixir and Phoenix. I'm kind of picking a little bit up as I work with them. I think Elm's architecture behind the scenes is kind of based, I believe on Erlang’s process model, so the idea of a mailbox and sending messages and dealing with immutable state.

CHARLES:: Which is kind of ironically is very object oriented in a way, right? It's functional but also the concept of mailboxes and sending messages and essentially, if you substitute object for process, you have these thousands and thousands of processes that are sending messages back and forth to each other.

PHILIP: Yeah, that's right. It's like on a grand scale, on a distributed scale. Although I wouldn't say that I'm that far with Erlang, Elixir to appreciate the reality of that yet but that's what they say absolutely.

CHARLES:: Now, Phoenix and Elixir is a dynamically typed functional language. does it share the simplicity? One of the criticisms you had of Rails was that you couldn't fit it in your head. It was very difficult. Is there anything different about Elixir that kind of makes it a spirit cousin of Elm and the simple Ruby?

PHILIP: I think so, yes. Absolutely. I don't think it gets to the same level but I think it's in the right direction and specifically on the framework front, it was designed specifically... I mean, in a sense it's like the anti-type to Rails because it was born out of people's frustrations with Rails. José Valim was pretty much one of Rails top core committers. Basically, every Rails application I wrote at one period, at 80% of the code written by José Valim, if you included all the gems, the device and the resourceful and all the rest of it.

Elixir in many ways was born out of the kind of limitations of Ruby with Rails and Phoenix was also born out of frustrations with the complexity of Rails. While it's not as simple as say, Michel Martens' Syro which is like his web framework, which is a successor to Cuba if people have heard of that, it is a step in the right direction. I don't understand it but I certainly feel like I could. They have plug which is kind of analogous but not identical to Rack but then the whole thing is built out of plugs.

I remember Yehuda Katz give a presentation like 'The Next Five Years' and essentially about Rails 3.0. This is going way back and Phoenix is in some ways the manifestation of his desire to have like the Russian doll pattern, where you could nest applications inside applications and you could have them side by side and put them inside each other and things like that. Phoenix has this concept called umbrella applications which tells that, like Ecto is a really, really nice obstruction for working with the database.

CHARLES:: I see. It feels like, as opposed to being functional or static versus dynamic, the question is how do you generate your complexity? How do you cope with complexity? Because I think you touched on it at the beginning of the conversation where you thought that my problems are complex so the systems that I work with to solve those problems must necessarily also be complex. I think one of the things that I've certainly realized, kind of in the later stages of my career is that first part is true. The problems that we encounter are extremely complex but you're much better served if you actually achieve that complexity by composing radically simple systems and recombining them. To the commonality of your system is going to determine how easy it's going to work with and how well it can cope with complexity. What really drives a good system is the quality of its primitives.

PHILIP: Absolutely. After ElmConf, I actually invited Michel to come to my place in the Netherlands. He live in Paris but I think he grew up Buenos Aires in Argentina. To my amazement, he said, "Yes, okay," and we spent a couple of days together and there he talked to me about Christopher Alexander and the patterns book, where patterns and design patterns actually grew out of.

One of his biggest things was the code is the easiest part, like you've got to spend 80% of your time thinking deeply about the problem, like literally go outside, take long looks. I'm not sure if this is what Rich Hickey means with Hammock Driven Development. I've never actually got around to watching the talk.

CHARLES:: I think it's exactly what he means.

PHILIP: And he said like once you get at, the code just comes. I think Michel's work, you should really check it out. I'll send you a link to put in the show notes but everything is built out of really small libraries that do one thing and do it really well. For example, he has a library like a Redis client but the Redis client also has something called Nest, which is a way to generate the keys for nested hashes. Because that's a well-designed, the Redis client is literally just a layer on top.

If you understand the primitive then, you can use the library on top really well. You can embed Syro applications within Syro applications. I guess, there you also need the luxury of time and I think this is where maybe my role as VP of engineering, which is kind of my first role of that kind, comes in here which is when you're working on the commercial pressure, try to turn around to a business guy and say, "Yes, we'll solve this problem but can we take three weeks to think about it?" It's never going to happen --

CHARLES:: No.

PHILIP: Absolutely, it will never going to happen. Although the small things that I tried to do day to day now is get away from the computer, write on paper, write out the problem as you understand it, attack it from different angles, think about different viewpoints, etcetera.

CHARLES:: I think if you are able to quantify the cost of not thinking about it for three weeks, then the business person that you're going to talk to is their ears are going to perk up, right? But that's so hard to do. You know, I try and make like when we're saying like, "What technologies are you going to choose? What are the long term ramifications in terms of dollars or euros or whatever currency you happen to be in for making this decision?" I wish we had more support in thinking about that but it is kind of like a one-off every time. Anyway, I'm getting a little bit off track.

PHILIP: No, not at all. This is a subject I love to talk about because we kind of had a few a bit of turbulence because we thought, maybe we should get product people in, maybe we should get them a product team going and what I find was -- and this is maybe unique to the size of the company -- that actually made things a lot more difficult because you got too many heads in many ways. Sometimes, it's better to give the developer all of the context so that he can think about it and come up with the best solution because ultimately, he's the only one who can understand. I wouldn't say understands the dollars and cents but he understands the cost implications of doing it in efficient ways, which often happens when you're working in larger teams.

TARAS: One thing I find really interesting about this conversation is the definition of good is really complicated here. I've observed Charles work on microstates and I work with him, like I wrote a lot of the code and we got through like five or six iterations and at every point, he got better but it is so difficult to define that. Then when you start to that conversations outside of that code context and you start to introduce business into the mix, the definition of good becomes extremely complicated.

What do you think about that? How do we define it in a way? Are there cultures or engineering cultures or societal cultures that have a better definition for good that is relevant to doing quality work of this?

CHARLES:: That's a deep question.

PHILIP: Wow. Yeah, a really, really deep question. I think often for business, like purely commercially-driven, money-oriented good is the cheapest thing that gets the job done and often that's very short term, I think. As you alluded to Charles, that people don't think about the cost of not doing the right things, so to speak in our eyes and also, there's a huge philosophical discussion whether our definition of good as programmers and people who care about our craft is even analogous to or equal to a good in a commercial context.

CHARLES:: Yes, because ultimately and this is if you have read Zen in the Art of Motorcycle Maintenance, one of the things that Pirsig talks about is what is the definition of quality. How do we define something that's good or something that's bad? One of the definitions that gets put forward is how well something is fit to purpose. Unless you understand the purpose, then you can't determine quality because the purpose defines a very rich texture, a very rich surface and so, quality is going to be the object that maps very evenly and cleanly over that surface. When it comes to what people want in a program, they're going to want very different thing.

A developer might need stimulation for this is something that's very new, this is something that's going to keep my interest or it's going to be keeping my CPU max and I'm going to be learning a whole lot. A solution that actually solves for that purpose is going to be a high quality solution. Also, this is going to be fast. We're going to be able to get to market very quickly. It might be one of the purposes and so, a solution that is fast and the purpose fits so it's going to be good.

Also, I think developers are just self-indulgent and looking for the next best thing in something that's going to keep their interest, although we're all guilty of that. But at the same time, we're going to be the ones maintaining software, both in our current projects and collectively when we move to a new job and we're going to be responsible for someone else's code, then we're going to be paying the cost of those decisions. We both want to minimize the pain for ourselves and minimize the pain for others who are going to be coming and working in our code to make things long term maintainable. That's one axis of purpose and therefore, an axis of quality.

I think in order to measure good and bad, you really have to have a good definition of what is the purpose of that surface is so rich but the more you can map it and find out where the contours lie, the more you're going to be able to determine what's good and what's bad.

TARAS: It makes me think of like what is a good hammer. A sledgehammer is a really good hammer but it's not the right hammer for every job.

CHARLES:: Right.

TARAS: I think what you're saying is understanding what is it that you're actually doing and then matching your solution to what you're actually trying to accomplish.

PHILIP: Yeah, absolutely and in my experience, we have a Ruby team building a Rails application. That's our monolith and then, we have a couple of Elixir teams with services that have been spun out of that. This isn't proven. This is just kind of gut feel right now and it is that Elixir is sometimes slower to develop the same feature or ship it but in the long term it's more maintainable. I haven't actually gotten dived into to React and all of the amazing frameworks that it has in terms of getting things up and running quickly but in terms of the full scale application, I still think 10, 11 years on, Rails has no equal in terms of proving a business case in the shortest time possible.

CHARLES:: Yeah. I feel very similarly too but the question is does your development team approach the problem as proving a business case or do they approach the problem as I want to solve the set of features?

PHILIP: Yes. Where I'm working at the moment, I started out just as a software developer. I guess, we would qualify for 37 signals or sorry... base camps definition of a calm company --

CHARLES:: Of a what company?

PHILIP: A calm company. Sorry. They just released a new book and called 'The Calm Company' and 'It Doesn't Have to Be Crazy at Work.' I was given in my first couple of months, a problem. It was business oriented, it had to be solved but it had to be solved well from a technical perspective because we didn't want to have to return to it every time. It was standardizing the way that we exported data from the database to Excel. You know, I was amazed because it was literally, the first time that I'd been given the space to actually dive in on a technical level to do that kind of stuff.

But I think even per feature, that varies and that sometimes challenging when handing the work on because you've got to say, "This fit. Literally, we're just trying to prove, whether if we have this feature, the people will use it?" versus, "This is a feature that's going to be used every day and therefore, needs to be at good, technical quality." Those are the tradeoffs that I guess, keep you in a job. Because if it was easy, then you would need anyone to figure it out but it's always a challenge.

What I like is that our tools are actually getting better and I think, with Elm for example, it's kind of major selling point is maintainability and yet, with Elm, there haven't been that many companies with Elm over a period of years that exists, that can live to tell the tale. Whereas, we certainly know with Rails applications have done well like Basecamp and GitHub. For sure, they can be super maintainable but the fact that it took GitHub to just moved Elm to Rails 5.0, I belief, the fact that it took them years and years and they were running off at fork of Rails 2.3, I think it shows the scale of the problem in that way.

You know, Phoenix also went through a few issues, kind of moving architectures from the classic Rails to a more demand driven design model. I think we're getting there slowly, zig-zagging towards a place where we better understand how to write software to solve business problems. I guess, I was really interested in microstates when you shared it at Wicked Good Ember because that to me was attacking the problem from the right perspective. It's like given the fact that the ecosystem is always changing. How can we extract the business logic such that these changes don't affect the logic of our application?

CHARLES:: Man, we got a lot to show you. It has changed quite a bit in the last two years. Hopefully, for the better.

TARAS: It's been reduced and it's almost a quarter of its size while maintaining the same feature set and it's faster, it's lazier, it's better in every respect. It's just the ideas have actually been fairly consistent. It's just the implementation that's evolved.

CHARLES:: Yeah, it's been quite a journey. It parallels kind of the story that we're talking about here in the sense that it really has been a search for primitives and a search for simplification. One of the things that we've been talking about, having these Ruby gems that do one thing and do it very, very, very well or the way that Elixir being architected has some very, very good primitives or Elm, the same kind of thing being spiritually aligned, even though on the surface, it might share more in common with Haskell. There's actually a deep alignment with a thing like Ruby and that's a very surprising result.

I think one of the things that appeals to me about the type of functional programming that is ironically, I guess not present in Elm, where you have the concept of these type classes but I actually think, I love them for their simplicity. I've kind of become disenchanted with things like Lodash, even though they're nominally functional. The fact that you don't have things like monoid and functors and stuff is kind of first class participants in the ecosystems, means you have to have a bunch of throwaway functions.

Those API surface area is very large, whereas if you do account for those things, these kind of ways of combining data and that's how you achieve your complexity, is not by a bunch of one-off methods that are like in Lodash, they're all provided for you so you don't have or have to write them yourself. That is one level of convenience but having access to five primitives, I think that's the power of the kind of the deeper functional programming types.

PHILIP: And Charles, do you think that that gives you the ability to think at a higher level, about the problems that you're solving? Would you make that link?

CHARLES:: Absolutely.

PHILIP: So, if we're not doing that, then we're actually doing ourselves a disservice?

CHARLES:: I would say so.

PHILIP: Because we're actually creating complexity, where it shouldn't exist?

CHARLES:: Yeah, I think if you have a more powerful primitive, you can think of things like async functions and generator functions, there's a common thread between async functions, generator functions, promises arrays and they're all functors. For me, that's a very profound realization and there might be a deeper spiritual link between say, an async function and an array in the same way that there's a deep spiritual link between Ruby and Elm, that if you don't see that, then you're doing yourself a disservice and you're able to think at a higher level. Also, you have a smaller tool set where each tool is more powerful.

PHILIP: You did a grit, I think it was a repository with a ReadMe, where you boiled down what people would term what I would term, the scary functional language down to a very simple JavaScript. Did you ever finish that? Did you get to the monads?

CHARLES:: I did get to the monads, yeah.

PHILIP: Okay. I need to check that out again. I find that really, really helpful because I think one of Evan's big things with Elm is he doesn't use those terms ever and he avoids them like the plague because I think he believes they come tinged with the negative experiences of people trying Haskell and essentially getting laughed at, right?

CHARLES:: Yes. I think there's something to that.

TARAS: But we're doing that in microstates as well, right? In microstates documentation, even though microstates are written completely with these functional primitives, on the outside, there's almost no mention of it. It's just that when you actually go to use it, if you have an idea, one of the thing that's really powerful with microstates is that this idea that you can return another microstate from a transition and what that will do is what you kind of like what a flat map would do, which is replace that particular node with the thing that you returned it with.

For a lot of people, they might not know that that's like a flat map would do but a microstate will do exactly what they wanted to do when it didn't realize that's actually should just work like that. I think, a lot of the work that we've done recently is to package all things and it make it powerful and to access the concepts that it is very familiar, something you don't need to learn. You just use it and it just works for you.

CHARLES:: Right but it is something that I feel like there's unharvested value for every programmer out there in these type classes: monads and monoids and functors and co-functors or covariant functors, contravariant functors, blah-blah-blah, that entire canon. I wish there was some way to reconcile the negative connotations and baggage that that has because we feel kind of the same way and I think that Evan's absolutely right. You do want to hide that or make it so that the technology is accessible without having to know those things. But in the same way, these concepts are so powerful, both in terms of just having to think less and having to write less code but also, as a tool to say, "I've got this process. Is there any way that could it be a functor? If I can find a way that this thing is a functor, I can just save myself so much time and take so many shortcuts with it."

PHILIP: And in order to be able to communicate that, or at least communicate about that, you need to have terms to call these things, right? Because you can't always just refer to the code or the pattern. It's always good to have a name. I'm with you. I see value in both, like making it approachable, so the people who don't know the terms are not frightened away. But I also see value in using the terms that have always existed to refer to those things, so that things are clear and we can communicate about them.

CHARLES:: Right. definitely, there's a tradeoff there. I don't know where exactly the line is but it would be nice to be able to have our cake and eat that one too. We didn't get really to talk about the type versus dynamic in the greater context of this whole conversation. We can explore that topic a little bit.

PHILIP: Well, I can finish with, I think the future is typed Erlang. Maybe, that's Elm running on BEAM.

CHARLES:: Whoa. What a take? Right there, folks. I love it. I love it but what makes you say that? Typed Erlang doesn't exist right now, right?

PHILIP: Exactly.

CHARLES:: And Elm definitely doesn't run on BEAM.

PHILIP: I don't know if I'm allowed to say this. When I was at this workshop with Evan, he mentioned that and I'm not sure whether he mentioned it just as a throwaway comment or whether this is part of his 20-year plan but I think the very fact that Elm is designed around like Erlang, the signal stuff was designed around the way Erlang does communication and processes, it means I know at least he appreciates that model. From my point of view, with my experience with Elixir and Erlang in production usage, it's not huge scale but it's scale enough to need to start doing performance work on Rails and just to see how effortless things are with Elixir and with Erlang.

I think Elm in the backend would be amazing but it would have to be a slightly different language, I think because the problems are different. We began this by saying that my story was a little different to the norm because I went back to the dynamic, at the dark side but for example in Elixir, I do miss types hugely. They kind of have a little bit of a hack with Erlang because they return a lot of tuples with OK and then the object. You know, it's almost like wrapping it up in a [inaudible].

There are little things and there's Dialyzer to kind of type check and I think there are a few projects which do add types to Erlang, etcetera. But I think something that works would need to be designed from the ground up to be typed and also run in the BEAM, rather than be like a squashed version of something else to fit somewhere else, if that makes sense.

CHARLES:: It makes total sense.

PHILIP: I think so. I recently read a book, just to finish which was 'FSharpForFunAndProfit' is his website, Scott Wlaschin, I think. It's written up with F# but it's about designing your program in a type functional language. Using the book, you could probably then just design your programs on paper and only commit to code at the end because you're thinking right down to the level of the types and the process and the pipelines, which to me sounds amazing because I could work outside.

CHARLES:: Right. All right-y. I will go ahead and wrap it up. I would just like to say thank you so much, Philip for coming on and talking about your story, as unorthodox as it might be.

PHILIP: Thank you.

CHARLES:: Thank you, Taras. Thank you, David.

TARAS: Thank you for having us.

CHARLES:: That's it for Episode 113. We are the Frontside. This is The Frontside Podcast. We build applications that you can stake your future on. If that's something that you're interested in, please get in touch with us. If you have any ideas for a future podcast, things that you'd like to hear us discuss or any feedback on the things that you did here, please just let us know. Once again, thank you Mandy for putting together this wonderful podcast and now we will see you all next time.

Guests:

In this episode, Amanda Hickman and Amberley Romo talk about how they paired up to get the safety pin, spool of thread, and the knitting yarn and needles emojis approved by the Unicode Committee so that now they are available for use worldwide.

They also talk about how their two path crossed, how you can pitch and get involved in making your own emojis, and detail their quest to get a regular sewing needle approved as well.

Resources:

This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC.

Transcript:

ROBERT: Hello, everyone. Welcome to Episode 112 of The Frontside Podcast. I'm Robert DeLuca, a software developer here at the Frontside and I'll be your episode host. With me as co-host is Charles Lowell. Hey, Charles.

CHARLES: Hello, Robert. Good morning.

ROBERT: Good morning. This is an exciting podcast. Today, we're going to be discussing writing a proposal to the Unicode Committee, getting it accepted and rejected. This is basically making emojis which I think is really awesome. We have two guests today who have an amazing story, Amanda Hickman and Amberley Romo.

Thank you both for joining us. You two have an amazing story that I would really love to dive into and we're going to do that today. It's about basically creating your own emoji and getting that accepted so everybody can use that and I think that's super, super cool, something that I've always kind of wanted to do as a joke and it seems like that's kind of where your stories began, so you two want to jump in and start telling? I think Amanda has a great beginning to this.

AMANDA: Sure. I mean, hi and thanks for having me. I don't know where to begin and really for me, this starts with learning to sew my own clothes which is an incredibly exasperating and frustrating process that involves a lot of ripping stitches back out and starting over and Instagram was a really big part of me finding patterns and finding other people who are sewing their own clothes and learning from the process.

I wanted to be able to post stuff on Instagram and it started to drive me absolutely crazy, that there's emojis for wrenches and nuts and hammers and there are no textile emoji. The best I could find was scissors which is great because cutting patterns is a place where I spend a lot of time procrastinating but that was it. I knew a woman, Jennifer 8 Lee or Jenny who had led a campaign to get the dumpling emoji into the Unicode character set. I knew she'd succeeded in that but I didn't really know much more about how that had worked.

I started thinking I'm going write a sewing emoji. I can do this. I can lead this campaign. I started researching it and actually reached out to Jenny and I discovered that she has created an entire organization called... What was that called? She's created an entire organization called Emojination, where she supports people who want to develop emoji proposals.

CHARLES: Before you actually found the support system, you actually made the decision that you were going to do this and you found it. You know, from my perspective, I kind of see emoji is this thing that is static, it's there, it's something that we use but the idea that I, as an individual, could actually contribute to that. I probably, having come to that fork in the road would have said, "Nah, it's just it is what it is and I can't change it." What was the process in your mind to actually say, "You know what? I'm actually going to see if I can have some effect over this process?"

AMANDA: It definitely started with a lot of anger and being just consistently frustrated but I knew that someone else had already done this. It was sort of on my radar that it was actually possible to change the emoji character set. I think that if I didn't know Jenny's story and it turned out I didn't know Jenny story at all but I thought I knew Jenny story but if I didn't know that basic thing that that somebody I knew who was a mere mortal like me had gone to the Emoji Subcommittee of the Unicode Consortium and petition them to add a dumpling emoji, I am sure that I wouldn't bother. But I knew from talking to her that there was basically a process and that there were a format that they want proposal in and it's possible to write them a proposal. I knew that much just because I knew Jenny.

I think at that point, when I started thinking about this, the Emoji 9 -- I should be more of an expert on that actually, on emoji releases but a new release of emoji had come out. There were a bunch of things in that release and it got a little bit of traction on Twitter. I knew that the Unicode Consortium had just announced a whole new slate of emoji, so I also was generally aware that there was some kind of process by which emoji were getting released and expanded and updated.

ROBERT: That's interesting. Do you know when that started? Because it seems like Apple started to add more emojis around like iOS 7 or something but it was pretty static for a while right? Or am I wrong?

AMANDA: I actually am tempted to look this up but the other piece that is not irrelevant here is that at the time, I was working at a news organization called BuzzFeed that you may have heard of --

ROBERT: Maybe, I don't know. It sounds kind of familiar.

AMANDA: I do feel like people kind of know who they are. I was surrounded by emoji all the time: in BuzzFeed, in internet native of the highest order and we had to use emoji all the time and I had to figure out how to get emoji into blog post which I didn't really know how to do before that. I can put them on my phone but that was it. I was immersed in emoji already. I knew that there was a project called Emojipedia, that was a whole kind of encyclopedia of emoji.

One of my colleagues at BuzzFeed, a woman named Nicole Nguyen had written a really great article about the variation in the dance emoji. If you look at the dance emoji, one of the icons that some devices use is this kind of woman with her skirt flipping out behind her that looks like she's probably dancing a tango and then one of the icons that other character sets use and other devices use is a sort of round, yellow lumping figure with a rose in its mouth that you sort of want to hug but it's definitely not to impress you with its tango skill. She had written this whole article about how funny it was that you might send someone this very cute dumpling man with [inaudible] and what they would see was sexy tango woman.

I think there was some discussion, it was around that time also that Apple replaced the gun emoji with a water gun. There was some discussion of the direction that the various emoji's face. One of the things that I learned around that time was that every device manufacturer produces their own character set that's native to their devices and they look very different. That means that there's a really big difference between putting a kind of like frustrated face with a gun pointing at it, which I don't really think of it as very funny but that sort of like, "I’m going to shoot myself" is very different from pointing the gun the other way which is very much like, "I'm shooting someone else," so these distinctions, what it means that the gun emoji can point two different ways when it gets used was also a conversation that was happening. None of that answers your question, though which is when did the kind of rapid expanse of emoji start to happen.

ROBERT: I feel like the story is setting in the place there, though because it seems like there's a little bit of tension there that we're all kind of diverging here a little bit and it's sort of driving back towards maybe standardization.

AMANDA: There's actually, as far as I know, no real move toward standardization but the Unicode Consortium has this committee that actually has representatives from definitely Apple and Microsoft and Google and I forget who else on the consortium. Jenny 8 Lee is now on the consortium and she's on the Emoji Subcommittee but they actually do get together and debate the merits of adding additional emoji, whether they're going to be representative. One of the criteria is longevity and I tend to think of this as the pager problem. There is indeed a pager emoji and I think that the Unicode Consortium wants to avoid approving a pager emoji because that was definitely a short-lived device.

CHARLES: Right. I'm surprised that it actually made it. Emoji must be older than most people realize.

AMANDA: My understanding is that very early Japanese computers had lots emoji. There's a lot of different Japanese holidays that are represented in emoji, a lot of Japanese food as well are represented in emoji, so if you look through the foods, there's a handful of things that haven't added recently but a lot of the original emoji definitely covered Japanese cuisine very well.

ROBERT: I definitely remember when I got my first iPhone that could install iPhone OS 2, you would install an app from the App Store that then would allow you to go toggle on the emoji keyboard but you had to install an app to do it and that's kind of where the revolution started, for me at least. I remember everybody starting to sending these things around.

AMANDA: But if you look at Emojipedia, which has a nice kind of rundown of historical versions of the Unicodes, back in 1999, they added what I think of as the interrobang, which is the exclamation/question mark together and a couple of different Syriac crosses. Over the years, the committee has added a whole series of wording icons and flags that all make sense but then, it is around, I would say 2014, 2015 that you start to get the zipper mouth and rolling your eyes and nerd face and all of the things that are used in conversation now -- the unicorn face.

ROBERT: My regular emojis.

AMANDA: Exactly.

CHARLES: It certainly seems like the push to put more textile emoji ought to clear the hurdle for longevity, seeing there’s kind of like, what? Several millennia of history there? And just kind of how tightly woven -- pun intended -- those things are into the human experience, right?

AMANDA: Definitely. Although technically, there's still no weaving emojis.

CHARLES: There's no loom?

AMANDA: There's no loom and I think that a loom would be pretty hard to represent in a little 8-bit graphic but --

CHARLES: What are the constraints around? Because ultimately, we've already kind of touched on that the emoji themselves, their abstract representations and there are a couple of examples like the dancing one where the representation can vary quite widely. How do they put constraints around the representation versus the abstract concept?

AMANDA: You don't have to provide a graphic but it definitely kind of smooths the path if you do and it has to be something that's representable in that little bitty square that you get. It has to be something representable in a letter-size square. If it's not something that you can clearly see at that size, it's not going to be approved. If it's not something you can clearly illustrate at that size in a way that's clearly distinct from any other emoji and also that's clearly distinct from anything else of that image could be, it's not going to be approved. Being able to actually represented in that little bitty size and I don't know... One of sort of sad fact of having ultimately worked with Emojination on the approval process is that we were assigned an illustrator and she did some illustrations for us and I never had to look at what the constraints were for the illustration because it wasn't my problem.

ROBERT: Sometimes, that's really nice.

AMANDA: Yes, it's very nice. I ended up doing a lot of research. What made me really sad and I don't want to jump too far ahead but one of things that made me really sad is we proposed the slate and the one thing that didn't get approved was the sewing needle and it also didn't get rejected, so it's in the sort of strange nether space. That's kind of stuck in purgatory right now. I did all this research and learned that the oldest known sewing needle is a Neanderthal needle so it predates Homo sapiens and it's 50,000 years old.

CHARLES: Yeah. Not having a sewing needle just seem absurd.

AMANDA: Yeah. We have been sewing with needles since before we were actually human being.

ROBERT: That's a strong case.

AMANDA: Yes, that's what I thought. If I sort go back to my narrative arc, I wanted to do a sewing needle and started researching it a little bit --

CHARLES: Sorry to keep you interrupting but that's literally the one that started this whole journey.

AMANDA: Yes, I wanted a sewing needle and I really wanted a sewing needle. I did a little research and then I reach out to Jenny and to ask her if she had any advice. She said, "You should join my Slack," and I was like, "Oh, okay. That's the kind of advice." She and I talked about it and she said that she thought that it made more sense to propose a kind of bundle of textile emoji and I decided to do that. She and I talked it through and I think the original was probably something closer to knitting than yarn but we said knitting, a safety pin, thread and needle were the ones that kind of made the most sense.

I set about writing these four proposals and one of the things that they asked for was frequently requested. One other thing that I will say about the proposal format is that they have this outline structure that is grammatically very wonky. They ask you to assert the images distinctiveness and they also ask you to demonstrate that it is frequently requested. I found a couple of really interesting resources. One, Emojipedia which is this sort of encyclopedia of emoji images and history maintains a list of the top emoji requests. I actually don't know how they generate that list or who's requesting that and where but I think it's things that they get emailed about and things people request in other contexts and sewing and knitting, I've done on that list and I started compiling it in 2016.

ROBERT: To be a part of the proposal process, to show that it is requested, without that resource, you just start scouring Facebook and Twitter and history and shouting to people like, "I really want this emoji. Why it didn't exist?" That seems pretty hard.

AMANDA: Actually our proposals all have Twitter screen shots of people grousing about the absence of knitting emoji and yarn emoji and sewing emoji. I know that Emojipedia, they do a bunch of research so they go out and look at based what people are grousing about on Twitter. They look at places where people are publicly saying like, "It's crazy that there's no X emoji," and that's part of their process for deciding what kinds of emojis people are asking for. Their research was one resource but we took screenshots of people saying that they needed a safety pin emoji and that was part of making the case.

One of the things that I found as I was doing that research was that, I guess at this point it was almost two years ago, when the character set that included the dumpling emoji came out, there was a bunch of grousing from people saying, "Why is there not a yarn emoji?"

There was a writing campaign that I think Lion Brand had adopted. Lion Brand yarn had put in this tweet saying like, "Everyone should complain. We needed a yarn emoji," but it doesn't matter how much you yell on Twitter. If you don't actually write a proposal, you're not going to get anywhere. I had been told that the Emoji Subcommittee, they're really disinclined to accept proposals that had a corporate sponsor, so they weren't going to create a yarn emoji because Lion Brand yarns wanted them to create a yarn emoji.

ROBERT: Right, so it was like counter-peer proposal.

AMANDA: Right. But as I was digging around the other thing I found was this woman in... I actually don't know if you're in Dallas or Austin but I found Amberley, who also put a post on Twitter and had started a petition, asking people to sign her petition for a yarn emoji proposal or a knitting emoji. I don't remember if it was a yarn emoji or a knitting emoji but I found her petition and reached out to her to ask if she was interested in co-authoring the proposal with me because she had clearly done the work. She actually had figured out how the system worked at that point. I think she knew who she was petitioning, at least.

I reached out to Amberley and we worked together to refine our proposal and figure out what exactly we wanted to request. I think there were a bunch of things that were on the original list like knitting needles, yarn and needles. I think crocheting would have been on the original list. We were sort of trying to figure out what was the right set of requests that actually made sense.

ROBERT: So then, this is where Amberley stories comes in and it is interesting too because she has entirely different angle for this. Maybe not entirely different but different than outright. This kind of ties back to the word software podcast mostly. It kind of ties back to the software aspect, right?

AMBERLEY: Yeah. I think, really they're kind of separate stories on parallel tracks. My motivation was also two-fold like Amanda's was, where I started knitting in 2013 and I had a really good group of nerd friends with a little yarn shop up in DC, like a stitch and ditch group --

ROBERT: I love it.

AMBERLEY: It was a constant sort of like, where's the insert emoji here, like where's the yarn emoji? Where's the knitting emoji? And we would sort of sarcastically use the spaghetti emoji because it was the most visually similar but that was something that was in the back of my mind but it teaches you a lot about yourself too because I was like, "Oh, this is like fiber art, not really an emoji. It's kind of technical, like on a tech space," and I didn't really connect that it was relevant or that I might have any power to change it. It just didn't occur to me at the time.

ROBERT: Interesting. I feel like a lot of people are in that similar situation or maybe not situation, even though you can make change on this.

AMBERLEY: Right, so my brain didn't even make it like, "Why isn't this a thing? let me look at how to make the thing." When that happened for me, Amanda mentioned using emoji and everything in the BuzzFeed space. I love how you explained BuzzFeed a while ago, it's my favorite description of BuzzFeed I ever heard. Something similar that happened for me was I was a software developer and in 2016, the Yarn package manager was released and that kind of turned something on in my head. That was like I'm seeing all these software engineers now be like, "Where's the yarn emoji?" and I'm like, "Welcome to the club."

ROBERT: "Do you want to join our Slack? We can complain together."

AMBERLEY: Right. It has been like a pretty decent amount of time, I'm semi-seriously ranting and complaining to my coworkers who were primarily male software engineers. I remember I went to [inaudible] in the Frost Bank Tower after work and was just like, "I'm going to figure out how this happens," and I spent a couple hours at the coffee shop. I found the Unicode site and I found their proposal process and their structure for the proposal and everything and I just started doing the research and drafting up a proposal specifically for yarn. Maybe it was a bit naive of me but to me it was like, "Okay, here's the process. I follow the process. Cool." I mean, you have to make a case and it has to be compelling and has to be well-written and it has to be supported and all that and that to me it was like, "Okay, there's a process.

At the same time, I did read about the dumpling emoji but I didn't connect it to Emojination and they had started the Kickstarter. We should talk about this later but I think the sort of idea the issue of representation on the committee and who gets to define language is really interesting but I saw that they had done the Kickstarter and there was a campaign aspect to it, so I ended up just building up this simple site so that if anyone Google, they would find yarn emoji. It's still up at YarnEmoji.com and that was how Amanda found me.

I got this random email, I sort of like had this burst of energy and I did all the research and I wrote the draft, sort of piecemeal, filling out the different sections of the way they have it outlined on the Unicode site and then I feel like a month or two went by and I had kind of not looked at it for a bit and then, I get this random email from this website that I almost forgot about. It was like, "Hey, I'm working on this series of proposals. If you're working on knitting or yarn or whatever, maybe we could work together," and I was like, "Well, that's sweet." Then she opened up this whole world to me. There's this whole Emojination organization, sort of 100% devoted to democratizing the process of language formation through creating emojis and so then, I got really into that. My primary motivator was yarn.

CHARLES: So what's the status of the yarn spool, those emoji right now?

AMBERLEY: The yarn, the spool of thread and the safety pin, they're all approved emoji for the 2018 released. Amanda and I are actually at the end. Amanda, a couple of months ago when I saw someone used the spool thread emoji for a Twitter thread -- you know how people will be like all caps thread and have a thread of tweets -- I saw someone do that just out of the blue. I was like, "Oh, my God. Is it out?" and the thing about these individual vendors, it sort of gets released piecemeal, so at the time Twitter have I think released their versions of this series of new emoji but others hadn't.

CHARLES: How does that work? Because you think the Twitter would be kind of device depending on what browser you're using, like if you're on a Windows or a Mac or a Linux Box, right?

ROBERT: -- Emoji set, right? I know Facebook does this too.

AMBERLEY: I'm painfully aware that Facebook does it because I can't use the crossed finger emoji on Facebook because it actually gives me nightmares.

ROBERT: I have to go look at this now.

AMBERLEY: Because it's so creepy-looking.

CHARLES: Okay. Also like Slack, for example is another. It's like a software-provided emoji set.

AMANDA: Right.

AMBERLEY: I'm not totally sure that Slack actually adheres to the standard Unicode set. I think it's kind of its own thing but I might be wrong about that.

AMANDA: Sorry, Slack definitely supports the full Unicode set. They also have a bunch of emoji that they've added that aren't part of the set.

AMBERLEY: Slack emojis?

AMANDA: Yes.

CHARLES: Yeah and then every Slack also has its kind of local Slack emoji.

AMBERLEY: Right.

CHARLES: But how does that work with --?

ROBERT: Okay, this crossed-finger Facebook emoji is... yes, I agree with you, Amberley.

AMBERLEY: Thank you. I had yet to find someone who disagrees with me about that.

AMANDA: I have never seen it before and I'm now like, "What is going on?"

CHARLES: Yeah, so how does it work if a vendor like Twitter is using a different emoji set? How does that work with cut and paste, like if I want to copy the content of one tweet into something else? Are they using an image there?

AMANDA: They're using an image. I think it's doesn't happen as much anymore but for a long time, I would often get texts from people and the text message would have that little box with a little code point in it and you were like --

AMBERLEY: More like an alien thing?

AMANDA: Yeah. Definitely, if you don't have the emoji character set that includes the glyph that you're looking at, you're going to get that little box that has a description of the code point and I think what's happening is that Twitter is using JavaScript or generally programming. There were air quotes but you can't see.

Twitter is using their software to sub in their emoji glyph whenever someone enters that code point. Even if you don't have the most up to date Unicode on your computer, you can still see those in Twitter. If I copy and paste it into a text editor on my computer, what I'm going to see is my little box that says '01F9F5' in it but if I get it into Twitter, it shows up. I can see them on Twitter but I can't see them anywhere else.

AMBERLEY: Damn, you really have the code point memorized?

AMANDA: No, I --

CHARLES: Oh, man. I was really hoping --

AMBERLEY: Oh, man.

ROBERT: You live and breathe it.

AMANDA: No, I'm not that compulsive.

AMBERLEY: We definitely have our emojis on our Twitter bios, though.

AMANDA: Absolutely.

ROBERT: If you see Amanda's bio, it's pretty great.

AMANDA: They started showing up on Twitter and I think that somebody in Emojination probably told me they were out and that was when I first started using them. Amberley might have actually seen it. It sounds like you just saw it in the wild, which is kind of amazing.

AMBERLEY: I saw it in the wild with this tweet thread and yeah, it's just [inaudible]. I was like, "Amanda, is it out?"

CHARLES: Yeah, I feel like I saw that same usage too, although I obviously did not connect any dots.

AMANDA: This last week, October 2nd -- I'm also looking things up. I'm just going to come to the fact that I am on a computer looking things up so I can fact check myself -- after they actually released their emoji glyph set, so by now any updated iOS device should have the full 2018 emoji, which in addition to a kind of amazing chunky yarn and safety pin, there's also a bunch of stuff. There's a broom and a laundry basket. There's a bunch of really basic, kind of household stuff that certainly belongs in the character set alongside wrenches and hammers.

AMBERLEY: I think one of the big ones too for this year was the hijab?

AMANDA: No, the hijab actually came out with a dumpling. Hijab has been available --

AMBERLEY: It's been up, okay.

ROBERT: So did it come with iOS 12 or 12.1? I don't know for sure. I just know --

AMANDA: I'm looking at it and it's 12.1. I really feel that I should be ashamed that I have used the internet and search for this.

AMBERLEY: I would say, I have no idea what their release numbers are.

AMANDA: [inaudible] as it appeared for the first time in iOS for 2018 with today's release of the iOS 12.1, Beta 2 for developers.

ROBERT: That is amazing. Do you get some kind of satisfaction -- like you have to, right? -- from people using the emoji and it's starting to make its way out there?

AMANDA: So much. Oh, my God, yeah.

AMBERLEY: I didn't really expect it, like saying that random tweet using this spool of thread for a tweet thread. I just thought and I just got so psyched. For me, I'm a knitter. I have knitter friends and it started with yarn and then really, Amanda and through Amanda, Jenny really sort of broadened my idea of what it all really meant. To think someone using it in the wild for a totally different application than I had ever thought of was like, "That's legit."

AMANDA: I definitely have a sewing emoji search in my tweet deck and sometimes, when I'm feeling I need a little self-validation, I'll go look over there and find people who are saying things like, "Why is there no sewing emoji?" and I'll just reply with all the sewing emoji, like it is part of my work in this life to make sure that not only do they exist but people know about them.

ROBERT: That is awesome. I would do the same thing, though to be honest. You'll be proud of that.

AMANDA: Totally.

ROBERT: Were there any hitches in the proposal process? I know we're kind of alluded to it but the thing that you started off one thing, Amanda didn't make it. Right?

AMANDA: I know.

ROBERT: So how did that process happen from you two meet each other and then going through the actual committee and the review process and then being accepted. What would that mean?

AMANDA: The process is actually incredibly opaque. We wrote this whole proposal, a bunch of people edited it, which is one of the other nice things about collaborating with Emojination. There was a bunch of people who are just really excited about emoji and the kind of language making that Amberley was talking about. There's a whole bunch of people who just jumped in and gave us copy edits and feedback, which was super helpful and then, there was a deadline and we submitted it to the committee and it actually shows up in the Unicode register which is also a very official kind of document register. I was a little excited about that too but then they have their meeting.

They first have a meeting and there's like a rough pass and the Emoji Subcommittee makes formal recommendations to the Unicode Consortium and then the consortium votes to accept or reject the Emoji Subcommittee's recommendations. It's a very long process but unless you're going and checking the document file and meeting minutes from the Unicode Consortium meetings, you'll never going to know that it happen.

AMBERLEY: -- You know someone connected through there because one of the things in our first pass, it wasn't that it was rejected. It was that we needed to modify something. We do have art for knitting needles with yarn because at one point, I think we weren't totally sure that a ball of yarn would be visually distinct enough in this emoji size to look like yarn and so, we had put it with sort of knit piece on knitting needles.

AMANDA: Oh, that's right. There was a tease of a little bit of knitted fabric.

AMBERLEY: Right and I think that, probably through Jenny or the people actually in the room, the feedback I remember is that there is a crocheter in the room who was like, "Yeah, why isn't there a yarn emoji but knitting needle?" so there was a little bit of like that was how I think we ended up from knitting needles with a fiber piece to ball of yarn, maybe.

AMANDA: I think that sounds right. I'm actually sure of that. It's just all coincide with my recollection. There were some things that they had questions about and that happened really fast because I feel like we had a couple of days and they have stuck to our guns and said, "No, we're only interested in knitted bit of fabric." Also, we worked with an illustrator and went back and forth with her because the initial piece that she had illustrated, I feel like the knitting needles were crossing in a way. That was not how knitting works and so, there was a little bit of back and forth around that as well.

But then once they decided that the they like the thread, yarn and safety pin, we're going to move to the next stage. I actually had to go back and look at the minutes to find out that the two reasons that they didn't move the sewing needle on to the next stage is when they thought it was adequately represented by the thread, which I wholeheartedly disagree with and they thought it wasn't visually distinctive. That's so much harder because a sewing needle, which is really just a very fine piece of metal with an eye at the end, you get down to a really small size and it is maybe a little hard to know what you're looking at.

But I think there's such a big difference between the static object which is the spool or the thread which represents a lot of things and is important and the needle, which is the active tool that you use to do the making, to do the mending, to do the cobbling.

CHARLES: Yeah. I'm surprised that it almost isn't reversed when certainly in my mind, which I think is more culturally important in terms of the number of places which it appears, it's definitely the needle as being kind of... Yeah.

AMANDA: Yeah and I think that the thread and yarn, they're important and I think that the decision to have a ball of yarn rather than a bit of knitting makes sense because there's a lot of things that you can use a ball of yarn that aren't just knitting and they think that --

AMBERLEY: And it's the first step too that doesn't exclude anyone in the fiber art community.

AMANDA: But there's so many things like in sutures and closing wounds, you're not using a little spool of cotton thread for that or polyester thread and stuff like embroidery and beadwork, you might be using thread or fiber of some sort that started on a spool but you might not. Embroidery floss was not sold in a spool and there's all these places where we use needles and all kinds of different size and you don't always use thread.

Sometimes, you're using yarn. Sometimes, you're using leather cord. Sometimes, you're using new bits of, I would say Yucca. You're using plant fibers to do baskets and in all of these different practices, that process of hooking it through the eye and sewing it is how it's actually made. It still sort of mystifies me why they haven't accepted it but they also didn't reject it, which is really interesting. I don't know how many other emoji are sort of sitting in this weird nether space because sometimes they just reject them outright. I think there was a proposal for a coin that they just said no.

ROBERT: They were a like, "A coin?" That would be [inaudible].

AMANDA: Oh, God.

ROBERT: They have to add one for every --

AMANDA: [inaudible].

CHARLES: Literally, the pager of 2017.

AMANDA: Exactly.

CHARLES: So what recourse is now available to you all and to us, by extension, to get the sewing needle?

AMANDA: I'm actually working on a revised proposal and I've been trying to figure out what are all the arguments that I'm missing for why sewing and the needle are not adequately represented by the thread and yarn. A bunch of things that a friend of my named, Mari who's half-Japanese, half-American but lives in Guatemala and does all this kind of arts in textile work, pointed out that there's a whole holiday in Japan devoted to bringing your broken needles and thanking them for their service. I thought that was really cool.

I've been trying to formulate what are all of the arguments for the necessity of both a needle and a spool. If anybody has interesting ways to phrase that, I would love for arguments.

CHARLES: Yeah but it's hard to imagine the arguments is just anything being more compelling than the arguments the you just laid out that you named about seven context: shoemaking, medicine, different fibers where the needle operates completely and totally independent of the thread. It's looming so large in kind of our collective conscious like holidays, being dedicated to them, except I think the Cro-Magnon pager, which is made out of stone, I believe, the being the artifact that pre-dates...

AMANDA: There's the idiom landscape as well. Things like finding a needle in a haystack, that has a very specific meaning --

ROBERT: And for puns. I've been resisting saying a pun this whole time.

AMANDA: Oh, share your pun with us.

AMBERLEY: Yeah, you have to say it.

ROBERT: Well, you could say that trying to get this through the committee is like threading a needle. Butchered but --

AMANDA: There's a biblical quote about getting into heaven -- a camel through the eye of a needle. I forget actually how it...

CHARLES: To thread a camel through the eye of a needle than for a rich man to get into heaven.

AMANDA: Exactly and there's this sort of do-re-mi, saw, a needle pulling thread. There are all these places where it's about the needle and somebody had --

CHARLES: It's primarily ancient.

AMANDA: I know.

CHARLES: It is the prime actor. Maybe, this is a good segue into kind of talking about the makeup of the committee and the decision making process and these kind of what seem like very clear arguments might not be received as such.

AMANDA: I certainly don't want to say anything bad about anybody on the committee.

CHARLES: No, no, I don't think that there's anything bad. I think that being receptive to things which are familiar to us versus with things that aren't is a very natural human thing and it can be interesting to see that at work and at play.

AMANDA: The Unicode Consortium is also evaluating all of these requests for whole language glyphs sets. Lots of languages and lots of character sets that are kind of obvious, like there has to be a sort of like character set like there has to be an Arabic character set but there are a lot of languages that have been left out of that because they're very small minority languages or they are historical languages, where the actual writing is no longer written the same way but there's historical reasons to be able to represent those characters.

One of the reasons why the Emoji Subcommittee cares about what gets into the formal character set is that everybody has to accommodate it and there's already been, I think some grousing. People start to moan and groan about how there's too many emoji, then it's too hard to find things.

CHARLES: And there's no take backs.

AMANDA: There's no take backs. You can't undo it. The committee is made up of representatives from a lot of tech companies primarily, although there's a couple of other kind of odd additional folks on there. I do try to find the committee list and I can find it right now.

AMBERLEY: I have it from Emojination. I don't know if it's up to date but Oracle, IBM, Microsoft, Adobe, Apple, Google, Facebook, Shopify and Netflix. The other voting members --

ROBERT: Shopify?

AMBERLEY: Yeah, right? The others being the German software company, SAP and the Chinese telecom company, Huawei and the Government of Oman.

AMANDA: Yeah, the Government of Oman is a fascinating one. I don't think they're the ones that are biting us on this. Especially for those tech companies, every time the emoji character set adds 10 or 12 emoji, they don't have to accommodate it on their devices. They have to put illustrators on it, they have to deal with everyone saying that the crossed fingers emoji in Facebook looks like I-don't-even-know-what.

AMBERLEY: Hey, Amanda.

AMANDA: It's all your fault. There's a whole process and there's non-trivial work associated with every single new emoji, so wanting to put the brakes on a little bit and be intentional about where and when they apply that work, it doesn't seem crazy to me. I just want them to approve the thing that I want.

AMBERLEY: I like the way that Emojination captures it. I looked at their website earlier and actually, they take it down but their goal quote "Emojination wants to make emoji approval an inclusive representative process." There has to be a process. There's overhead involved but looking at the makeup of the decision makers are not a trivial question.

CHARLES: Right. This is a great example like [inaudible] metaphor but these little artifacts, these emojis are literally being woven to the fabric of a global culture and certainly, everybody uses them and they become part of the collective subconscious. It does seem like very important to be democratic in some way. It sounds like there is a process but making sure that everyone has a stake.

AMBERLEY: Yeah.

ROBERT: What was the reason that they gave for not accepting the needle and thread? Was it like a soft no? You said it's like just hanging out, not really rejected but not accepted. We're going to drop a link in the show notes for the proposal and your GitHub and everything. I'm looking at the PDF that was put together and it seems like it was all a package deal like we talked about. How do they just draw or they just take like a lawyer would, just like draw or cross it out like, "Well, no but we'll take the other ones."

AMANDA: Yes, basically. What they did is they need to discuss and I don't know how long they've been meeting but they need to discuss all of the proposals that have been supplied by a particular deadline and --

ROBERT: That sounds painful.

AMANDA: Yeah, I mean, it's --

ROBERT: Just imagine the power of thinking about emojis.

AMANDA: One of the things that they rejected, I think because there's the smiling poo face. Somebody wanted a frowning poo face and they rejected that. There's a bunch of things that actually do get rejected. I don't know if they've been really care about a smiling poo face versus a frowning poo face.

ROBERT: What about an angry one?

AMANDA: We got all the feelings of poo.

ROBERT: We got important work to do here.

AMANDA: But they go through when they're trying to figure out. I think to some degree, you want to get them when they're not tired but I think the status that it's listed right now is committee pushback, so they've set it aside until we have some concerns. We're not going to reject it outright but we're not really sure why this isn't adequately represented. Then their most recent meeting, they just kind of passed on reconsidering it, which is fine because I think I was traveling and my proposal is not done. I really want to make sure that I have consolidated every imaginable argument in one place so --

ROBERT: And make it strong as possible.

AMANDA: Yeah. If people want to help the other thing that would be amazing is any and all idioms that you can think of, especially ones that are not in English or European languages, idioms in Central European languages, idioms in Asian languages that refer to needles, either translations of the kind of classic, 'finding a needle in a haystack,' but also any idioms that are kind of unusual and specific to a culture outside of what I have experience with would be amazing for making the case, so this is an international need.

ROBERT: Do they need any specific or actionable feedback or do they just say, "We're going to push back on this. We're just not quite sure?"

AMANDA: The two things that we're in the minutes -- there are minutes and they publish the minutes to Unicode.org -- were it was not visually distinct, which is not totally crazy. We actually worked with an illustrator to get a different image. The first image was almost at 90 degrees. It was kind of straight up and down and it is a little hard to see and the second is --

ROBERT: Especially, because it's thin.

AMANDA: The second image is actually a kind of stylized needle because it's fairly a little fatter and the eye is bigger but it's much more distinctively a needle. I'm hoping that that will also convince them but you have to be able to tell at a very small size that it's a needle. The other thing that they said was that sewing was already represented by the thread, that we didn't need thread and needle but it was literally one line in the minutes that referenced that and then it sort of like, "Did you have somebody in the room or not?" and so, if there is somebody on the committee who is willing to tell you really what their concerns were, then you have some sense of what they're looking for and why they're pushing back.

When you can very much see in the earliest emoji character sets that I have a hammer and I have a wrench and I use them but there's these very conventionally male tools. We have all of the kind of office supplies but all of homemaking and housekeeping and textile production, none of them were there until very, very recently. I think it does reflect the gender of the people who've been making these tools, that sewing and knitting weren't important enough as human practices to be included in this glyph set.

AMBERLEY: I guess, that's non-trivial to mention because that wasn't an argument that I made in my original yarn draft and Amanda and Jenny sort of pushing to open it up to this whole slate of craft emoji. I didn't realize until they brought that up. I took a stroll through pretty much the whole slate of emoji and you can count on almost one hand the number that represented the creative endeavors or sort of more traditionally known as creative things like camera or painting palette and stuff like that. It was extremely limited.

AMANDA: I think they have stuff like that. I think there's a few different variations on the camera and then there's painting palette and that's it.

AMBERLEY: Oh, there's the theater mask.

AMANDA: Oh, that's right. There is the theater, the happy and sad --

AMBERLEY: And I don't know it exactly and I haven't read the minutes like Amanda has but I think and I hope that that was a particularly compelling piece of that argument.

AMANDA: I think they definitely heard it.

AMBERLEY: Yeah.

CHARLES: Opening it up then, what else is coming in the way of craft? It sounds like this is historical but these pieces are being filled in not only with the work that you all are doing but by other emoji which you're appearing.

AMANDA: Yes.

CHARLES: And are you in contact with other people who are kind of associated with maybe craft and textiles and other kind of what you're labeling historically creative spaces?

AMANDA: I don't think there are anymore with a possible exception. Someone's working on a vinyl record proposal which I think is great.

CHARLES: Yeah, that's awesome.

ROBERT: Antiquated, though.

AMANDA: Maybe not, I don't know.

AMBERLEY: Take a stroll through the Emojination Slack and people discussed that.

AMANDA: Yeah. If you click at Emojination.org, the whole Airtable database is on there. There's not a lot of other creative ones. A friend of mine got really bent out of shape about the lack of alliums and wrote a whole slate proposal for leeks and scallions and garlic and onions.

ROBERT: Oh, there is a garlic one, right?

AMANDA: No. I mean, there is --

AMBERLEY: Actually, I'm looking at the Unicode page for current emoji candidates. They first get listed as... I forget the exact order. They become draft candidates and then provisional candidates or vice versa but I don't see any pending further creative ones but garlic and onion are on there.

AMANDA: Yes.

ROBERT: That makes my Italian a little happy.

AMANDA: I think there's some prosthesis, the mechanical leg and the mechanical arm, a guide dog --

AMBERLEY: Ear with hearing aid, service dog.

AMANDA: Yeah, there's a good chunk of interesting things that have been left out. I guess they've been approved by the subcommittee but are still waiting on final approval by the Unicode Consortium.

ROBERT: Okay. What are the next steps that we can do to help push the thread and needle proposal through it. You mentioned a couple things like coming up with idioms that are in different languages and whatnot but how can we contact you and push this effort and help?

AMANDA: That's such a good question. I don't even know. I mean, I am [email protected] and you're totally welcome to email me if you want to help with this and I will --

ROBERT: That's a great domain, by the way.

AMANDA: Unfortunately, there's no information about velociraptors anywhere on that site.

ROBERT: That's the way it should be.

AMANDA: But also, if you're excited about working on emoji proposals, Emojination is an incredibly great resource and folks there, including me actually will help you identify things that are on other people's wish lists that you could work on if you just want to work on something and we'll help you refine your proposal if you know what you want and we'll help you figure out whether it's worth putting the time in or not and how to make it compelling. You can definitely check out Emojination.org. I think there's a path to get on to the Slack from there.

AMBERLEY: Oh, yeah. The Slack and the Airtable.

AMANDA: Yeah.

ROBERT: It sounds like there's a whole community that was born out of this, where everybody is trying to help each other and collaborate and get their shared ideas across.

AMANDA: Definitely and there's a woman, Melissa Thermidor who is fantastic, who actually is a social media coordinator. It's her actual title but she works for the National Health Service in the UK and was tasked with getting a whole series of health-related emoji passed. There's a bunch of things that she's --

AMBERLEY: Is she's the one doing blood.

AMANDA: She's doing blood.

AMBERLEY: That's a good one.

AMANDA: Because there's a lot of really important health reasons why you need to be able to talk about blood and getting blood and blood borne illnesses and --

AMBERLEY: That one was listed on the emoji candidate page or blood donation medicine administration.

AMANDA: Yeah.

ROBERT: That's really interesting, so she works for the government, right? and that was part of her job to do that?

AMANDA: Yes.

ROBERT: That's awesome, actually. I love that.

AMANDA: Yeah, I think the drop of blood, the bandage and the stethoscope are the three that are in the current iteration, which is interesting because the existing medical emoji were the pill and that gruesome syringe with a little drops of fluid flying off of it, which do not do a lot to encourage people to go to the doctor.

ROBERT: No, not at all.

AMANDA: So a few more, we're welcoming medical emoji.

ROBERT: You have a GitHub. Is that where you're still doing for the follow up and the prep work for the sewing emoji?

AMANDA: Yeah, that's probably the best place. I do have a Google Docs somewhere but that's probably a better place to connect even than my ridiculous Velociraptor email. The GitHub --

ROBERT: But it's still awesome.

AMANDA: It is awesome. I won't lie. I'm very proud of it. I am AmandaBee -- like the Bumble Bee -- on GitHub and the sewing emoji, the original proposals are there and I will make sure that there is information about how to plug into the revised needle proposal there as well. You guys are a tech podcast, so if people want to just submit suggestions as issues on that repository, that's awesome. We'll totally take suggestions that way.

ROBERT: That would be pretty rad. Well, I appreciate you two being on the podcast. I love hearing your stories and how it ended up converging in parallel tracks but it end up achieving the same goal. Still unfinished, right? Let's see if we can help push this over the finish line and get it done because I would really like to see a needle. I could definitely use that in many of my conversations already now, making all kinds of puns. Thank you, Amanda for coming on and sharing your story.

AMANDA: Thanks for having me.

ROBERT: And thank you, Amberley for also coming on and sharing your story. This was super awesome.

AMBERLEY: Yeah and thank you for connecting us to finally have a voice conversation.

AMANDA: I know. It's great to actually talk to you, Amberley.

CHARLES: Oh, wow, this is the first time that you actually talked in audio?

AMANDA & AMBERLEY: Yeah.

ROBERT: We're making things happen here. The next thing we have to do is get this proposal through and accepted.

AMANDA: Yes.

CHARLES: You've converted two new faithful sewing and needle partisans here and I'm in.

AMANDA: Awesome.

ROBERT: I know you've already gotten, what? Three through accepted?

AMANDA: Yeah.

ROBERT: We talked about that, it's got to be really awesome. I think I want to try and jump in and get that same satisfaction because a lot of people use emojis.

AMANDA: Exactly.

CHARLES: It definitely makes me think like you look at every single emoji and there's definitely a story. Especially for the ones that have been added more recently, there's a lot of work that goes into every single pixel. That represents a lot of human time, which I'm sure you all know, so thank you.

AMANDA: Thanks for having us on.

AMBERLEY: Yeah, thank you guys.

ROBERT: Cool. That is the podcast. We are Frontside. We build UI that you can stick your future on. I really love this podcast because it wasn't necessarily technical but had a lot of interesting conversation about how to work with a proposal and probably make a bigger impact than any of us with software, just because the sheer reach that emojis have are insane and the fact that you can influence this process is new to me and really cool, so I hope a lot of other people learn from that too.

If you have any feedback that you would like to give us on the podcast, we're always open to receive feedback. We have our doors and ears open, so if you like to send an email at [email protected] or shoot us a tweet or DM us at @TheFrontside on Twitter. We'd love to hear it.

Thank you, Mandy for producing the podcast. She always does an amazing job with it. You can follow her on Twitter at @TheRubyRep. Thanks and have a good one.

In this episode, Robert, Charles, and Wil talk about the whys and hows of accessibility, as well as what makes single page applications special, why they are they harder for accessibility, and frameworks that can do this for you.

Resources:

This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC.

Transcript:

ROBERT: Hello everyone. Welcome to The Frontside Podcast. This is Episode 111. I'm Robert DeLuca, a software developer here at the Frontside and I'll be your episode host. Today, we're going to be discussing accessibility in single page apps. With me as co-hosts are Charles Lowell. Hey, Charles.

CHARLES: What is up Robert?

ROBERT: And Wil Wilsman. Hey, Wil.

WIL: Yo, yo, yo, yo.

ROBERT: Sounds like we're ready to drop a disc track. We're not going to be dissing anybody here. We're going to be talking about helpful things with accessibility in single page apps. Before we get into the nitty-gritty of accessibility in single page apps because we're getting into some deep stuff, I think I want to cover a lot of 'how' because I know accessibility things are usually about why you should be doing it and then they touch on things like, "You should be using alt attributes for your images," but for single page apps, I think we need to go further.

CHARLES: It always ends up like, "Then draw the rest of the accessibility owl."

ROBERT: Yeah, here's your two circles and the alt-tags are your circles and then the rest of the freaking owl is focus management and everything else that comes with it. Before we get there, what is accessibility? I guess, if we trim the giant umbrella down a little bit from everything that is accessibility that can be physical space things, like wheelchair accessible ramps or things like that, what about just technology?

WIL: People need assistive tech to interact with technology such as switches or keyboards and obviously screen reader is a big one but when it comes to the software itself, you could even talk about colors and people who are color blind, so red might not be red to everybody.

ROBERT: Speaking from experience there?

WIL: Yeah. I'm colorblind. Red is brown to me.

CHARLES: Things like hearing and all of it, right? It really is like just designing it in such a way that it can be used by as many people as possible.

ROBERT: Right.

WIL: And that includes your mom who may not be the best with technology but she still needs to pay your bills online or something.

ROBERT: Exactly, yeah and in for context to listeners that might not know, my mom is 100% blind, so it's kind of where it comes from.

CHARLES: But my mom is not but she has all kinds of problems.

WIL: Yeah, same with my mom.

CHARLES: That also falls under the category of accessibility, right?

ROBERT: Absolutely.

CHARLES: Right, accounting for age and culture.

ROBERT: We're blending into the why of accessibility, which is perfect. One of the things that it's such a good segue because what people are starting to realize and I think why accessibility is really starting to catch wind and get some traction is because a lot of people that grew up in the technology age were open in using technology a lot. Our parents probably did not use technology heavily. That's definitely the case for my dad. He still has a flip phone and he says, he's a low tech man living in a high tech world and just refuses to pick up technology but those are the people that just didn't use technology. But now we have a lot of people that grew up with technology and use a lot and they're aging into more disabilities and they're going to need that accessibility, which I think is really interesting to think about because that's a lot of buying power if you're just going to start moving in that needs in accessibility, right?

CHARLES: That's true. I know I may need glasses pretty soon, so colors and fonts were going to be heck a lot more to me in the next five years than they have over the last 20.

ROBERT: Yup, exactly and that's going to be huge for that and that's one piece of the why, so what are the other reasons that you'd pick up accessibility other than people saying that it's morally correct. I don't like starting off conversations for accessibility because it's the thing that you should be doing.

WIL: I think it goes even to my user experience like power users that don't like using the mice or mouse. That's me. I really prefer to just use my keyboard for everything. When the new Firefox browser came out and I couldn't navigate through the menus the way I was used to, I went back to Chrome.

ROBERT: That's interesting.

CHARLES: I still have not found a good workflow for navigating tabs with the keyboard without just kind of twisting my wrist all out of shape. You have to share that with me but again, that's another thing. That's an impediment that sits between me and the application, that actually --

ROBERT: Which is really interesting, you're getting into a keyboard navigation and focus management, which is really the crux of accessibility for screen reader users and switch users.

CHARLES: What I'm hearing is that in this case, including good keyboard and focus management in your application, e.g. making it accessible to screen readers, you at the same time, enabling your power users. I think a great point is that by introducing these very low friction workflows, you're actually going to be enabling other parts of your customer base and not just catering to one, that there are ripple effects throughout your system.

ROBERT: It may set it for everyone.

WIL: Yeah, not only people who need it but people who don't know they need it.

ROBERT: Yes, exactly. Think about the WCAG spec as user experience guidelines. They're not telling you how to implement a thing specifically for a screen reader. They're telling you how to implement it in a way that works for everyone, regardless of what ailment they have. It could be a temporary ailment. It could be a permanent ailment, where they have to use a screen reader or any kind of ailment that you can think of. They're not considering just one use case. It's a broad thing that shows how you can make your application better for everyone. I think that's a better way to look at the WCAG spec than I need to read through this and make sure that this auto complete works for a screen reader.

If you look at the guidelines, they're not telling you just first screen reader. They're telling you how to make it work for a switch, someone who is colorblind or who is using a dictation software. I kind of tend to look at that as UX guideline, which really helps me build a better app overall because when you nail down that user flow, everyone benefits from it because it's pretty well thought out at that point.

CHARLES: I like that too and I think that thinking of it as user experience and making sure that you have a complete user experience is a good way to think about it because it kind of separates the concerns of, "I've got HTML but is my application really HTML or is it a set of workflows and the data over which those workflows operate?" It really forces you to think my application is not a set of React opponents or web components or Ember components or what have you but really, there's a deep structure to it and it makes you kind of shine a light on that deep structure and try to map its surface. Then, if you really, really know it and you capture it, then you can represent it in any medium.

I think it's just a win not just for one niche group of users but also for all of our users and then also for your future users that you don't have because your application is designed better and is going to work. Who knows? Maybe there's some new interface or some new medium, some new device that comes out that hasn't even been invented yet. But if you really have a strong internal representation of what your application is, you're going to be able to be the first to move to that market.

ROBERT: Right and like I said earlier, people are starting to aid in the needing accessibility thing. If you need a dollar to justify it, that's going to be a big reason coming up. As a part of the new WCAG 2.1 spec, there is a zoom. I forget what this access criteria number is but the new criteria says your app basically needs to be responsive. That kind of maps directly to what you said earlier, Charles which is like you're going to need glasses soon and fonts and being able to zoom. It's going to be really important.

We see that just pop up everywhere. To give a counter example of not just for accessibility like you need it for glasses but the other side of that could be like when we give demos on low resolution projectors or screencast or anything, when we zoom the screen, the apps just still be usable and you should still be able to demo it and that's just something that you have both sides to that, where accessibility kind of works out for everybody. People on the call probably don't need glasses but to see that tiny screen that's being shared, you should be able to assume that.

CHARLES: Right. I'm just kind of restating what you said but I want to make sure to call it out explicitly because it was a definitely an aha moment for me, that you basically if you build yourself an accessible website, you build it so that it works on projectors and mobile devices too, so you kind of killed two birds with one stone and you don't have to make a special effort because zooming in the screen is tantamount to viewing it on a phone or viewing it on a tablet.

WIL: Yeah, we mentioned it earlier about accessibility and physical space and one of those things is being able to access something from anywhere no matter the device that person is using.

ROBERT: Yes. That's a lot of the 'why' of accessibility and I think we did a pretty good job of staying away from the more argument because morally we don't know if it's the right thing to do but a lot of the times it's not in front of you, it's hard to do and I want to make sure that you don't feel bad if you're not building accessibility in your apps. It's not easy. I don't sort of like people get up and say that, "Accessibility is easy. Just do this," because it's not --

WIL: If it was easy, everybody would do it.

ROBERT: Exactly. I always come back to accessibility being just like UX -- user experience -- because if it were easy, everybody would have a really great user experience too. It's a hard thing to boil down and to simplify it, right?

CHARLES: Right and like every other aspect, the kind of nonfunctional requirement, I say nonfunctional requirement that's a little bit of a contradiction terms but it becomes very hard if you haven't done it from the beginning. But if you didn't start with an accessible app, you weren't thinking about that or you inherited the app or it just wasn't on your radar for whatever reason. If you got a codebase that's two years old, going back and trying to make it accessible, it was extremely hard. It's expensive, expensive, expense.

WIL: Yeah, it's going to cost more in terms of money and time to added it after the fact.

ROBERT: Exactly. I spent a year on helping on Visa Checkout and there were some accessibility considered in the designs but a lot of the time my feedback wasn't just like, "Yeah, sprinkle some ARIA attributes on there and you're good." It was like, "Do we really need a carousel here to represent your list of carts because it's really hard to navigate around that?" A lot of the times, it ends up boiling up to is this the best way we can represent this data? Is this the best way we can navigate and build this user flow? A lot of times --

CHARLES: And if the answer is no, it's so painful, right?

ROBERT: Yeah, exactly, so all that work that was put into building that carousel and all the components that built that carousel together is thrown out because it's just not a pattern that should be used there. Doing after the fact is really hard and really expensive and usually ends up in refactoring anyways.

CHARLES: I just want to point that out because a lot of people find themselves in that situation, where they're staring down at a pretty big cost. Now, the reasons why your app may not be accessible or many and good and you shouldn't feel bad, if you're finally in that situation.

ROBERT: I gave a talk at Nodevember a couple of years ago called Accessibility Debt and it's just like any technical debt. Your apps going to have it. It's okay. Don't get beat up about it, especially if anybody is trying to beat you up over it, don't listen to them. It really is just like any other kind of technical form of technical debt. It's something that you'll have to deal with and it is just something you have to work through. It's not world ending. It's just another problem to work through.

What are some things like everybody usually talks about like the things you can do, the basics for making your app more accessible like using all its attributes and instead of doing a div with an on-click handler, just use a button or don't overuse ARIA attributes. Are there any other things that I missed there like the basics of accessibility?

WIL: The biggest is the HTML structure. Screen readers and other assistive tech were built with the standards in mind, so if you're doing nonstandard things like putting divs in H1 and adding ARIA attributes within there, you're not going to have a great time.

ROBERT: Right or splattering ARIA roles all over the place, probably not a good idea. That will be harder to debug. Also fun fact, ARIA roles, while you can implement directly to the spec, you may still have bugs across the different screen reader combinations or assistive tech combinations, so that's fine.

CHARLES: Keep it super simple is what I'm hearing, like use semantic markup. If you're going to introduce a custom button, still make sure that it's a button.

ROBERT: Use the platform.

CHARLES: Yeah, use the platform. Don't fight the platform. Probably the best example of that is people implementing their own select boxes. That's the classic example.

ROBERT: Wil and I, our lives has centered around that for a little while. It's so true. Usually, it's the first thing people go to grab to reimplement because selects are just ugly. I think Firefox has the ability for you to style select options now like you can change the color and the font but you can't style that. The pop up that comes, usually that's the system dialogue, which a lot of designers don't really like. That's usually the first thing that people go to implement and that's usually actually the first thing that stop somebody from signing up. A lot of sign up forms that I see, if your date of birth is in a select format, that probably will hinder somebody that uses assistive tech from signing up.

CHARLES: Yeah. You just basically bounced that entire person. The thing is people don't appreciate the cost. This gets into the whole concept of accessibility that is how much money would that person or those group of people have actually brought into your site versus the cost that you spent redoing that select box. You might be thinking, "It only took this developer actually two weeks," but when you actually look at the actual cost over the course of your application, you're not factoring that into the decision to go with custom select box. Just in our experience, it's just the truly low cost option per quality of experience, that tradeoff there is almost invariably going with platforms select, right?

ROBERT: Yeah. Your secret sauce and the best UX that you're going to provide is not going to be nicely styled select box and seriously, a battle that I had to fight a lot was if you really want to implement a custom widget, decide if this is what you want to spend your time on because custom widgets aren't just quick and easy things you implement. You're not going to implement the select that's fully accessible across all the AT combos in a couple of days.

CHARLES: It's a lot of work.

ROBERT: Yeah. You're going to fall down on a huge rabbit hole.

CHARLES: Yeah. It is just you're committing to that work over the lifetime of your application.

ROBERT: Exactly, if you can maintain that now. If you implement custom check boxes and custom radios and custom input that of content edible for some reason, I think --

WIL: I think what I see is like custom date pickers --

ROBERT: Oh, I just had a rant about that.

CHARLES: Date pickers are hard and there's not really a good option. You just have to open your eyes to the true cost.

ROBERT: Right, exactly. That's what I always try to explain, just like you have ownership over this now and you now have to maintain this and you can't regress.

CHARLES: Right and if you do regress, it's your neck that gets choked.

ROBERT: Yes. A good way to put it. We've talked a lot about things that are just general accessibility but nothing specific to single page apps. I do want to say like a lot of other things that people recommend is like if you're using React, use like the JSX ES1 plugin to help you analyze if you're writing any JSX that might not be accessible or use like HTML_CodeSniffer or aXe to statically analyze the DOM that you have.

Those tools are great. I'd see a lot of people [inaudible] those things as like, "Look, this automated checkers says I'm inaccessible so I'm inaccessible," and that's not the case, especially in a single page apps. You can have 100% passing automated checkers but your app also could be 100% broken and why is that?

WIL: I don't know.

ROBERT: I'm wobbling the router question here.

WIL: Yes. I guess that would just be due to routing. In single page applications, they handle their own routing, whereas static web sites and whatnot, the routing is handled despite URLs and the browser and reloading pages.

ROBERT: Right. There are probably other major differences between single page apps but the biggest thing is the routing is managed by the client, the JavaScript and on a server-rendered page, you click a link, it's going to rerender the entire page for that next page and the screen reader and the browser know exactly what to do with that, to put the focus back to the top of the page and start working through the page for you. But with single page apps, you're just replacing a piece of the DOM and the screen reader has no idea of what's going on.

An automated checker cannot check for this. They cannot tell you if your routes are accessible. The reason I'm bringing this up is because this has never talked about. I don't see in accessibility talks and this is the thing that actually is most broken and makes your app pretty much unusable to anybody that's using assistive tech. There's ways that people if they're savvy can navigate around it but if they don't know what's going on, they're going to think, "All right, I pressed this button and nothing happened, so I'm just going to leave now because it's not working."

CHARLES: There's a great video that you point to me, I believe a guy from the UK who has recorded a bunch of his experiences using websites and apps --

ROBERT: Yes, I want to dig it up and put it in the show notes.

CHARLES: Yeah. Those are great to watch because you'll really get it.

ROBERT: Yeah and that's a really unique case because he's really savvy. I believe, I could be wrong but I think he might have said that he definitely work in tech somehow --

CHARLES: Right. He knows the workarounds and he knows these things but in some of the cases it's like, "If I didn't work in tech, there's no way that I would be able to use this website."

ROBERT: Yeah. This is kind of the crux of like if you ever listen to anybody that is an accessibility consultant, they'll say, "You will never ever be able to automate accessibility," and this why I tie accessibility so much to user experience, would you ever have a user experience test that can tell you in a binary fashion? Yes or no, that your app has a great user experience? No, because it's pretty subjective.

CHARLES: Of how does your users feel? right?

ROBERT: Yeah.

CHARLES: You can rate, "My users feel great."

ROBERT: Accessibility is like that because there's a lot of context that you have to carry around. It's all about context. When I transition from this page, the next thing does the user have enough context of where they're coming from to where they're going to be able to operate on that page. Is there enough information to achieve the task they want? That is pretty much the crux of why there is no binary yes or no for that because it's contextual. It varies from person to person but you do your best to make sure that that works and that you provide enough information to do something.

That's kind of a single page app as a crux. This is why we have a philosophy of testing as a whole. We don't test components individually because again, you can make all of your components individually accessible there but as a whole, they might not work together because you're not providing enough context on an entire page. We did this in one of the apps that we work on, which is open source, so we can link to the PR that Wil wrote for this and I wrote an entire routing documentation around our philosophy and the different things that we tried. How did we manage the focus in that application? I'm kind of just going to lob it over to you Wil since you did the work. Do you want to give context of the holdings and how that all kind of came together?

WIL: Yes. One of the features of the app is these panels. It's like this three panel system. When you click an item in a list, a third panel pops up and this goes back to the context thing where if you're using a keyboard, you can't see the screen. You click an item in a list, how do you know that third panel popped up? The solution isn't for a component. The component can't be responsible for this. The list can't focus the item. It opens or vice versa, so this is definitely an application concern, where we needed to check against the route and see whether or not, the pane is opening or was already open or wasn't open before and when this pane opens for the first time, it will just focus it and that gives a lot of the context that the user needs when they click it, like they click an item in the list, it focuses this third pane and they're on a third pane.

ROBERT: Right. We didn't even just focus on the entire div. We focused on the heading of the thing that you selected.

WIL: Yeah because focusing the div, it might read something off but not all the time. The main thing we're focusing on that third pane opens is the heading to let them know that the item you click, you are now on that page and reading that heading. This is the same thing that would happen if you loaded up that page statically and the screen readers would usually just focus on that first heading.

ROBERT: Right. That helped a lot. For a little bit more context, the middle pane is like kind of a master detail thing going on here and in the middle pane there that we have, it's an infinite scrolling list. You have different things there and one of them is like a package. If you select the package and without the focus management and focusing on that heading, you would have to go through every single package that's in that list which could be a thousand of them before you actually get over to the pane that you just opened because source order. You have to go through each one of those.

WIL: And it's the same thing is true for power users, not just screen readers. It's like if they want to use tab once that pane open, they have to tab through the entire list.

ROBERT: Exactly. It wasn't keyboard accessible and it made it really hard to navigate around with a keyboard because the focus just wasn't being managed. There was a lot of work that we did there. I want to focus on the routing situation there because if you can't navigate around with a keyboard in your single page app, like you click a link and it's not selecting the next thing that should be focused and you provide the right amount of context, your app probably won't be usable without a lot of trial and error to a user, which depending on what your product is, they may not have a lot of patience for trial and error, right?

WIL: Yeah.

ROBERT: It's really important to try and nail down the routing situation and there are some frameworks and things out there that can do it. In the app that we're talking about, it's React app and we use React router but we don't actually really hook into the router to handle that and that's because React router doesn't provide very much information.

WIL: Yeah. You can think of React router as more of an outlet system, where your routes can render anything, anywhere on the page. That's kind of dangers of accessibility and that's kind of the reason that React router can't handle accessible things very well because at any point, the route can just change the button on the page to look like something else.

CHARLES: Yeah. It just pop in and pop out. There's no deeper model, right? There's not --

WIL: Yeah, there's no tree.

CHARLES: Right, there's the internal state --

WIL: Like a component tree, yeah.

CHARLES: The internal state of what is happening is completely opaque. You can only analyze the second and third order effects of the React tree.

ROBERT: Right and especially if you have nested route components, it's really hard to determine. One of the things that I've seen people do is focus on mount, which is a very naive approach because what if you have three nested routes, they're all going to focus on mount and the last one that mounts wins and that's a focus for which nobody wins.

CHARLES: You all tried that, right?

ROBERT: Yes. That's part of the document that I wrote up. We tried three different approaches and we ended up landing on something because we were using React router that was more of like an application state thing, so we were checking props because we knew what the user flow was. We knew what the user, when they come to this page is trying to accomplish, so we're able to kind of figure out from where they came from or where they're going through props and focus the right things for them.

WIL: One of the examples, like we talked about the third pane opening and focusing the heading inside but what you know what happens when you close that third pane, you kind of lose contexts again, so we have to focus the previous list item that was active because if you focus back at the top of the list, they've essentially lost their place and the list of results. In that case, we couldn't use on mount. That list item is already mounted. We have to listen to props and we have to look at the route through these props to determine if that third pane was open. If it just closed and if an item in the list is active, it should have focus. It's a lot of logic going on. You have to really understand your app in order to make it a very good accessible app. You can't just sprinkle in ARIA attributes and focus on mount everywhere and think it'll work fine because you're accessible. You have to really know the flow.

CHARLES: Right. All those signposts point towards having a deeper application state, a deeper understanding of your application than just the render tree. At that point, it's too late and so by that, I'm definitely lobbying a Reach/Ember router.

WIL: Yes. We talked a lot about the React router and we can't really be too accessible with it but to create a React router to go out and there is now Reach router. Robert, have you heard any good things about that? You're the accessibility expert here.

ROBERT: I haven't played with it myself, so [audio glitch] things that were lost from the React router three to four transition, I think and also, it's actually accessible routing which is nice because it comes out of the box and you know how to implement it. I haven't played with it. I know Gatsby has implemented that for their V2, that's their default router now, which makes me really happy because a lot of static sites that were being built with Gatsby were very inaccessible and broken which made me sad but now, they're not with V2.

I haven't played with Reach router but one of the things that I think it provides which was missing from React router was transition hooks. It has a concept of like where you're going from and to for your routes, which really helps figure out what you need to focus. The one thing I will say about Reach router and I'm sure it's got to be configurable somewhere but I haven't really looked and the demos that I saw, they were just focusing the div of the content that's being rendered, like it kind of just wraps with a generic div or the tabindex="-1" and then focuses it. If you have a lot of content that's inside of that div, it probably will be confusing but at least, it manages the focus somewhere.

If you're now off in a no man's land, you have no idea what's going on, at least it focuses that. But if you want to really nail the experience to be better, I would recommend trying to figure out what you should focus inside of that route that you just transition to, what is the best thing. That's for React. There are other things out there for other frameworks, like I am on Ember accessibility team that's out there and we have Ember A11y, which just provide you a focusing outlet for you to be able to just drop in your app and then when the route changes, it does the same thing. It focuses the wrapping div of that outlet that was just rendered.

I want to emphasize more in talking about e-holdings work that we did that we were focusing the heading because that told you exactly where you're at and what package you're on or what thing you're on. You know the name of it and now you know how you can go and navigate through that.

CHARLES: But it's conceivable, so how would you do that with the Ember router? Would you just introduce some way to delegate down to a particular component? Like when I'm rendered into an outlet, it focus on this component?

ROBERT: That would be interesting. I actually don't know. It's been a long time since I've messed around with the Ember router and Ember accessibility. It's definitely a great first step and that's where it kind of came from. I think there needs probably some work done to help implement on what you want to focus. That's kind of where I was going with when we were first exploring the React router stuff and e-holdings was I wanted to have this like high order component that knew of your routing tree and it knew where you're coming from or where you're going.

Then from there, it would just tell you, this is the route that we're going to and there's need to be focus management done, like if this prop exists, then you can pick inside of that component that what you want to focus. It leaves it up to the implementor of what they need to focus but it could be probably just like a fallback to the general div because that's not bad. It's a good first step. There needs to be a little bit of work there to get that done probably.

CHARLES: Yeah. But it is worth pointing out that by starting from a position of having an externalized application state via the route structure, in an Ember application, you're starting 95% of the way there. An Ember application, at any point, you know where you are and during your transition, you know where you're starting and where you're going to end up and you know when you leave and you know when you got there.

ROBERT: Yeah. Having an Ember route pivot handler there, like where you're pivoting from and to is just so nice and it kind of made it click together a lot easier.

CHARLES: Right. Reach router looks interesting but as I understand, it's still couched in React components and it feels to me like this is a problem that ought to be solved, that ought to be framework agnostic. Because if I use something like Reach or I use some other routing library for another framework, some other tool might come out that I want to use and I shouldn't be locked in --

ROBERT: Right like what if I really like Redux little router, then I have to make a choice between Redux state that I like or accessibility.

CHARLES: Exactly and that feels like a false dichotomy to me. What I would want to be looking for is a platform independent or a framework independent routing library that really just helps you represent your application state and the concrete state in which your application is in at any moment and then, also be able to represent the transitions between those states fully and completely. Then if you have that, you could embed that into any framework.

ROBERT: Right. That would be really nice to have. Just like pull it off the shelf and help you out there.

CHARLES: If anybody is listening who wants something to do for the next 18 months, no one will thank you for 18 months but you had to get on that. We'll give you lots of thanks then.

ROBERT: [inaudible] compare with you. That would be awesome.

CHARLES: I would love to be part of that.

ROBERT: Yeah, that would be awesome. To kind of tie back to other things, I don't know too much about Angular but I'm sure there are solutions out there to help with Angular. I know Marcy Sutton used to do a lot of work in the Angular world, so I'm sure there's something out there that helps with that. I just don't know off the top of my head right now.

I wrote a Medium 'think piece'... No, I wrote a little medium blog post about how all of single page out for routers are broken. I was pleasantly surprised by Vue. Vue brings its own router and their router has a concept of before each, so before each route transition you can run some code and that really helped with implementing the live, the announcer pattern where you use ARIA live to announce something but even beyond there, if you wanted to dig in there, you could probably figure out where you're transitioning from and to and give that next route some kind of attribute that says, "This needs to be focused. Figure out what you need to focus there and inside that route, you can focus wherever you want." I thought that was a really awesome.

That's kind of the crux of what I was missing from React router. I wanted the concept of knowing where I'm coming from or where I'm going and I would help with everything but unfortunately, that kind of doesn't exists because they're just components. For better or for worse, they're just components.

CHARLES: The world is so much more than just components.

ROBERT: Yeah, a little bit off topic, I think it's kind of funny how React kind of just shoved everything into the Vue layer, just to make it all a component.

CHARLES: Yeah, it's very easy.

ROBERT: Yeah, until it's not.

CHARLES: Yeah, exactly. It's easy but it's not simple.

ROBERT: That's a lot of talking about how routes need to be done and what you can kind of do to manage focus. It's really about managing the context and how you can provide the most contexts. For somebody to operate on that information, can they complete this thing? What can you do to make sure that you've done this properly? What steps can you take to make sure that you actually are accessible and that your routes work?

WIL: Manually testing those screen reader is probably the biggest thing, you know?

CHARLES: Yeah. I think the biggest thing is really watching someone who uses assistive tech on a regular basis use your application and then trying to use it yourself.

WIL: Yeah.

ROBERT: Right. Yeah, I'll definitely --

CHARLES: If you can get a little bit of this yourself but then it's kind of like someone who is never used a mouse before and trying to learn something new. What really helps is seeing someone who's good at it and see how their expectations are either being met or not being met.

ROBERT: Yeah, it's definitely the best way. If you can find somebody that actually relies on assistive tech, there's nothing that beats that kind of feedback. If they get to your app and they're really confused, I see some people that just dismiss it because they just don't understand. That is the best feedback you can actually get. If they don't understand what's going on, you have --

CHARLES: That's on you.

ROBERT: Yeah, that is going to be what happens for everybody that uses that. Well, maybe not everybody because everybody has different experiences but it's probably going to be a thing that pops up everywhere. But if you don't have access to people that are actually using assistive tech regularly and are pros at it, WebAIM provides really great tutorials for how to use a screen reader. If you're using a Mac, you have a screen reader built in and you can use that called VoiceOver. If you ever want to turn it on or turn it off, its command F5.

WIL: You might have to have that shortcut enabled, though.

ROBERT: Really? I'm pretty sure it's quite default.

WIL: I thought I had to enable mine but I could be wrong.

ROBERT: It's interesting.

CHARLES: Yeah. They got great tutorial too. It's like it notices the first time you turn it on, so it tries to help you navigate bullet lists and select boxes and input fields and check boxes and all kinds of good stuff.

ROBERT: Yeah. It gives you a little bit of a boot camp but WebAIM also helps with specific to website and stuff because one of the things that we ran into while working on the e-holdings project is they're transitioning from a native app to a web app and there were just things that you can do on a native app that you cannot do on a web app. Just keep that in mind also when you're testing, there's just things that will behave differently. Like you're not going to have a lot of crazy key shortcut commands like you're not going to press command F to get to the search box if your app has a prominent search box because that is going to clobber a bunch of other assistive tech key commands.

WebAIM is really a good help with giving you the tutorial of how to test a web app and many different screen readers, so they have it for JAWS, they have it for NVDA, they have it for VoiceOver iOS, they have it for TalkBack, which is the Android screen reader. There's a lot of really good resources there for you to start using as screen reader and test with your app.

I highly recommend using it with a screen off. You gain a lot of context by looking ahead of your cursor and --

CHARLES: Yeah. It's true.

ROBERT: -- The example that I usually give to people is like, "I worked on Visa Checkout. I went through that checkout flow pretty much every day for a year and eight months in, even then turning off the screen, I was still lost." Even though I knew what each screen was and what each component was there, I would find myself confused of like, what just happened and it's because sometimes, you'll get into something like you have a dropdown that sets the focus inside the dropdown and then the dropdown disappears from the DOM and your focuses in nowhere. You're like, "What is going? what am I doing. I don't even know where I'm at," and I turn the screen back on and I'm like, "Oh, now, I know what's going on."

CHARLES: Right. It really is a lot like interacting with your application as though you were interacting with Siri or Alexa but with a keyboard instead, instead of voice commands. That's an excellent point. We would understand if where would Amazon be if Alexa couldn't successfully navigate those situations. The counterpoint or even the flip side of that is if you model your web application in such a way that it can handle that type of serial interaction, instead of the highly parallel environment to being able to perceive huge amounts of information concurrently, like you can on a screen, if that were effectively serialized, that means you could represent your application through nothing but voice commands.

ROBERT: Yeah. Did you provide enough context for me to build this mental map is really what I'm going on to?

CHARLES: Yup.

ROBERT: Which I always thought was really interesting because I always wanted to know how my mom visualizes what a website looks like because it's wildly different than any of us. She's never been able to see what a website looks like. Does it look like a bunch of nodes and graphs and webs connecting to each other and how things pieced together? It's just a different way. But it's not only about screen readers, right? You can use a keyboard to navigate and that's definitely what we did with e-holdings is like can we tab through this [audio glitch] list, hit enter and go into the detail record, edit it, close it and go back and edit another one. Is that something that we can do with just a keyboard, not even a screen reader?

WIL: And with the keyboard, we're not talking about shortcuts to hit edit. We're talking about like tabbing over and hitting enter like people with accessibility issues would have to do.

ROBERT: Right and that's kind of a good segue into creating use cases. If you want to know if your app actually works, if your screen reader users or if your keyword users or if your dictation users are going to be able to navigate this app, create use cases. Things like actual user flows like how would somebody actually going to use this app? What task are they going want to complete? In that case, e-holding was like an electronic holdings management system for libraries. They probably want to get in there, add a couple of books or whatever you might have to their library and get out.

A use case could be like, "Can I search for this thing? I'm going to search for something specific. I'm going to go through the list, find the thing that I want. I'm going to add it, close the pane, go back and then remove one thing and can they complete that flow successfully without running into any issues or any blockers or any showstoppers." I can tell you before we did the routing management stuff, you would hit search and that was it. There was nothing else that you could do.

CHARLES: Yeah. They wouldn't even announce that anything can happen.

ROBERT: Exactly.

WIL: Yeah, and with these librarians, it's not necessarily a matter of they can't see the screen. It's just that they don't use the mouse because they're power users.

ROBERT: Right. They don't have any disabilities but that was essential to the workflow.

CHARLES: Yeah, exactly. Do that change. You just lowered the activation energy of that workflow by... What? three orders of magnitude?

WIL: Yeah, at least.

ROBERT: Let me click here right now. I can tab, now I can tab. Right now, let me click here and now I can tab, I can tab, I can tab. It's not as nice as just being able to completely do it through a keyboard. Through us making it super keyboard accessible, that also became super screen reader accessible and the people who use dictation were able to work through and get through the app and use it now, which was really cool. It really helps when you go for those things and create use cases to really figure out how a user is actually going to work through this app. That's the best way. Just get right through it. With Visa Checkout, we're like, "Can somebody buy something?" or if they don't have an account, can they sign up and buy? those were some of the use cases that we had because it turns out, those are actually pretty important to the business.

That all have been said, you also can test your components for accessibility individually because even at a smaller level, some of your components might have to manage focus. The best one I can think of is models. When you open a model, you should trap the focus inside of that model and you should be able to hit escape to close the model and when you close the model, it should go back to what triggered the model to be opened. These are all things that are individual that can be tested also.

But just because your components are individually accessible, it does not mean your application as a whole was accessible either. I don't want to paint the picture like, you don't have to care about your components accessibility individually because you do. It really does help but I think a lot of people miss the whole of the application, rather than individual with components.

CHARLES: Right. The components are the individual stitches but you have to follow the thread throughout the entire garment.

ROBERT: Right, exactly.

CHARLES: Man, what I really want to do is I want to find out how your mom visualizes website navigation and use that as a visualization technique.

ROBERT: That might be a fun webcast or something.

CHARLES: Yeah and then see like could we actually use it as a tool because I have to imagine, it's probably pretty simple. It's much simpler than what we think of when we think of a website because it has to be really condensed down to its essence.

ROBERT: Right. Yeah and [inaudible] users, they're not dumb. They have different ways. They know the gotchas. They know things that happen. They know there are different ways of getting trolled like my mom knows about focus jumping and she gets irritated what happens but she knows generally of what to do and it depends on her patience level. Like if it's focused jumping, she's like, "I don't need to use this thing. See you," and it's just not worth the frustration but there's different ways to navigate around like if you're a real power user, you might be able to recognize like the routing is inaccessible and you can navigate by headings or by regions or different landmarks. There's many different ways for users to navigate but to use those different navigation methods, you need to have a real strong coherent document structure. Your H1s have to actually be H1s and you have to have some things that they can navigate around to kind of work around those things.

It's interesting and basically, do everything you can to help those situations. If you can provide semantic markup and give it a proper structure. If you can do the focus management, it's going to help everybody. It really will. I did see when we went through the use cases and defining those things, we actually learned a little bit more about our product because we had to put ourselves in a different seat and think about it because you get real close to it. You go through that same flow nine million times and you pick up real power user things that you can do like, "I'll work on that. I got this. I'll click this button. All right, we're good." Somebody that's going through it for the first time and you put yourself in that seat, it kind of opens your eyes a little bit and makes the experience better for everyone. I think that's kind of the underlying tone there. That's the message.

WIL: Yeah, accessibility makes things better for everybody.

ROBERT: That was a lot of content thrown at you there. We covered what is accessibility and why you'd want to do that and then kind of like more the basics things and how automated tools are really helpful and they can help you pick out things like using improper roles and nested things but they're not going to be able to tell you if your application is truly accessible or not and never will, unless we get something like a headless screen reader, where you can write automated tests for in that fashion but you're not to get something that will just run over you app and go, "Yup, you're good."

CHARLES: Even so, it's a matter of user experience and that's not something you can get a thumbs up or a thumbs down to. When you as a user, it comes with application, you know when you see it but I would say until we have androids that accurately simulate human beings, I don't think we're going to actually have automated testing.

ROBERT: Yeah. There's a joke for accessibility consultants. It's like if you put four accessibility consultants in a room and tell them to give you an alt attribute for an image, you'll get four different alts.

We talked about the automated checkers, right? They'll not going to get everything for you and we talked about single page apps specifically in the routing and how we handled the routing in a React app and then how you can probably do it in an Angular app and how you can do it in an Ember app and Vue and different methods of how you can kind of attack that. We're definitely giving the link to the document that I wrote and the PR so you kind of see the real 'how' of how we did it because that would probably be helpful.

I think there was a lot of good information there, so I would like to thank both Charles and Wil for being awesome co-host on this and --

WIL: Thanks for being an awesome host.

CHARLES: Yeah, thanks for being an awesome host. I should say, you're welcome.

ROBERT: I tried, I tried. We are The Frontside. We do accessibility consulting and training. If that's anything that your team needs help with, we're more than happy to jump on a call with you to kind of figure out what your needs are and what you need to do. If you need a WCAG support statement, if you need to audit, if you just need to figure out what's the next steps for you to even do, we're more than happy to help you sort through that. To reach out for that, you can contact us at [email protected] or [email protected] or [email protected]. You can contact us at any different ways and we'll be more than happy to help you.

As always, thank you Mandy for producing the podcast. You're awesome and the next episode, what we're going to have is also still accessibility related and I'm really excited about this. It's about writing a proposal to the Unicode Committee and getting it accepted so basically, writing a proposal to get an emoji added and that's with Amberley and Amanda. They wrote three or four Unicode specs and actually got them accepted for, I believe it was sign language and deafness. That's really cool and I'm super excited for that because they'd be the first people that I've ever talked to that have actually created an emoji and gotten it accepted.

WIL: Yeah, it's pretty cool.

CHARLES: Yeah, that must feel great.

ROBERT: Yeah, it's going to be awesome. That's our next episode. If you have any ideas or comments or anything, you can tweet us at @TheFrontside on Twitter or you can contact us through any of the emails that I talked about earlier. We're always open to hearing feedback. Thanks for listening. Take it easy, everyone.