Herding Code

Herding Code

By Herding CodeTechnologyTech News
Download on the App Store

Herding Code episodes

  • Herding Code 219: Cake with Gary Park and Mattias Karlsson

    Episode 219

    Jon and Kevin talk to Gary Ewan Park and Mattias Karlsson about Cake, a cross platform build automation system with a C# DSL to do things like compiling code, copy files/folders, running unit tests, compress files and build NuGet packages.

    Download / Listen: Herding Code 219: Cake with Gary Park and Mattias Karlsson

    Show Notes:
    • Hello and Intro to Cake
      • (00:35) Gary and Mattias introduce themselves and explain what Cake is. Cake is a cross-platform build system that uses the Roslyn compiler and C# – like Grunt or Gulp, but using C# instead of JavaScript.
      • (01:48) Jon asks about the integrations and CI support. Gary says that’s a key feature: the build you run on your development machine is going to be as close as possible, if not identical to, the build you’re going to run on your CI server. Right now, Cake is set up with at least 10 online CI servers, which has the added benefit of providing them with a ton of badges for their GitHub readme.
      • (03:15) Jon says that he cloned the getting started repo and ran a script and some magic happened… but what exactly was that magic doing? Mattias explains that the bootstrapper (either build.ps1 or build.sh) fetches Cake from NuGet, then it launches the build.cake file.
      • (04:07) Jon asks if PowerShell on Linux will have an impact on Cake, or if the build.sh is simple enough that they’ll just stay with that. Gary says that the bootstrappers are very lightweight and normally don’t need to be changed (although you can if you want), so there’s no real advantage to moving to PowerShell since people who are building on Mac and Linux will probably prefer to just run a shell script.
      • (05:17) Kevin flips it around and asks if it matters for Bash on Windows. Gary says they’ve got this one covered, too, as they’ve already tested out Cake on Bash on Windows. Mattias says the hard part is the platform specific dependencies, and Gary agrees, saying that Cake is just a wrapper around the tools – there’s an expectation that the tools are either installed or available as a NuGet package.
      • (06:55) Jon asks if tools are well segmented per-project. Mattias confirms that everything is project specific – you can share if you want, but the default is that you can just clone a repository and build without thinking about dependencies.
    • Docker
      • (07:50) Jon asks how Cake works with Docker. Mattias talks about the preconfigured Docker images for Cake, which make it easy to easily test builds, handle integration tests, etc. He’s currently testing against Nano Server. Nano Server is very stripped down requires you to install a lot of prerequisites for Cake, so it’s nice to have a preconfigured Docker image for testing. Mattias talks about how versioning with tagged containers help them with integration testing. Gary talk about how they’re using Docker with Bitbucket Pipelines to really speed up their Travis build tests.
    • Cake Script
      • (10:25) Jon asks what a Cake script is – is it just C# code? Gary describes some of the additional build DSL features they’ve added, but other than that it’s just standard C# code.
      • (11:23) Jon muses about how there are build systems available in just about any programming language out there, but as a C# dev, it’s a lot easier to read a build script in C# code. Gary says that he’s a longtime user of PSake (a PowerShell based build system. The problem with that is that often his team members aren’t as familiar with PowerShell, so they don’t want to touch the build script. It’s a mental switch to go from coding in C# all day to switch to PowerShell to work on the build script. For the same reason it’s more comfortable to use a JavaScript based build script like Grunt or Gulp when you’re working on an Angular application and use FAKE when building an F# application, it’s just easier to work with a C# based build script when working on a C# project.
      • (13:30) Jon said that he opened the Cake file in Visual Studio code and saw there was an extension for syntax highlighting, and asks if there’s additional tools for IntelliSense. Gary talks about the current status of OmniSharp when running against Cake. Note that since this podcast was recorded, this has improved as explained in this blog post: How to debug a Cake file using Visual Studio Code.
      • (15:28) Jon asks about debugging a Cake script. Mattias says they’ve added a debug switch to the Cake exe, which waits for you to attach so you can debug. Jon says it sounds great to have debugging support for a build script. Mattias says it’s nice, but since your build scripts are often running on build server it’s also important to have good logging support.
    • Logging
      • (17:02) Jon asks what the logging support is. Mattias says there’s a built-in abstraction for logging as well as an exception handler, so to break the build you can just throw an exception. Gary say that the default is just to log to the build server like AppVeyor, but if you want to log to something like logstash you can. Mattias says that they log to standard output and standard error, and every build system integrates well with that.
    • .NET Core support, DI and Modules, Cross-platform Support
      • (18:06) Jon asks more about the .NET Core port. Mattias says they’re just about done, just working with integration tests. For the most part it’s pretty straightforward, but you can run into things like differences between kernel versions on Linux. The biggest issue has been waiting for dependencies to be available on .NET Core. In the past they’ve relied on Mono for Linux and Mac, and there are slight differences compared to the .NET Framework, so it will be good to be on Core CLR everywhere. (note: the port to .NET Core has since been completed).  
      • (19:52) Jon says he notices they’re using Autofac. Mattias says they use it for dependency injection throughout the codebase, but they’ve replaced it in some places for things like their module system.
      • (20:33) Jon asks how the module system works. Mattias says you just add a module folder in your tools and implement some attributes on your interfaces that indicate what your module should replace. (22:17) Jon asks about any issues they ran into with the .NET Core cross-platform port. Mattias says there are a lot of dependencies you don’t think about – for instance nuget.exe isn’t available cross-platform or on Nano Server, since it only has the Core CLR. Gary says they made a conscious decision not to implement .NET Core early, so they avoided some of the early adopter pain that some other projects ran into.
      • (23:47) Jon asks if they’re doing anything specific to handle platform differences. Mattias says that there are a few IFDEFs, but for the most part issues are around tools support, in which case it just won’t launch at all. Gary talks about platform specific criteria you can use in your build scripts to make platform specific decisions.
    • Cake Tasks, Parallel Tasks, Build System Integration, Unit Tests
      • (26:36) Jon asks how task names are used in a script. Mattias says that those labels are used to determine task dependencies. Gary says you can also use the labels as entry points which are specific to the build server.
      • (28:22) Jon asks if tasks are run in parallel. Mattias says it’s not currently multithreaded because logs would be difficult to follow. When you define a task, it’s just added to the graph, and nothing’s actually executed until you call RunTarget. Gary said there is an open issue to allow parallel execution of tasks, but they’re still working on a good story around debugging and logging. Mattias says it’s standard C# so there’s noting stopping you from running async tasks using the standard C# async methods.
      • (31:46) Jon says he sees support for several build systems and asks about how you integrate with them. Mattias explains how CI systems can call commandline options, and Gary says that all build systems have provisions for environment variables, and Cake provides a typed wrapper around that, so you get a strongly typed object from the build-specific provider to make informed decisions. For instance, when running on AppVeyor their script pushes the NuGet packages to the prerelease MyGet feed from the development branch and the official NuGet feed when running on release.
      • (34:09) Jon asks about assembly info patching and versions. Cake can update based on version info. It’s got file hash support, which is important for pushing to Homebrew.
      • (34:55) Jon asks about unit test integration. Gary says they have method aliases for calling common unit test frameworks, and you can easily extend for other frameworks. Unit test harnesses have published return values, so you can make informed decisions based on specific unit test results.
    • Publishing, Installers, Release Notes
      • (36:46) Jon asks about  publishing options. Cake can build the nuspec for you, you can use your own if you want, or on Core CLR you can use dotnet pack. Jon asks about some of the different ways NuGet is used as a deployment package. (38:42) Jon asks about other installers like WiX, NSIS, etc. Gary says that Cake is really just wrapping tools with strongly typed classes, which makes it easier to pass in property values to tools. There’s no real magic, it’s just bringing it up an abstraction layer, so you don’t have to remember the command semantics of each tool. Jon asks if, in addition to simplifying how you interact with tools, the abstraction makes it a little easier to switch between tools. Gary says that’s true, and describes how this works in practice.
      • (41:44) Jon asks about the release note parsing feature. Gary says there’s an included release notes parser, which can extract version numbers and bulleted release features from a markdown file assuming it’s in a known format. This allows you to stamp your assembly with the version number, and use the bulleted release notes in your NuGet or Chocolatey release notes. There are also Git Release Notes and Git Release Manager which can be used to generate release notes from issues in a milestone based release on GitHub. Gary says they use this with Cake releases, so just tagging a release gives them an end to end publish process which sets version numbers and writes release notes for all their release endpoints (NuGet, Chocolatey, GitHub release, etc.).
    • Addins
      • (44:54) Jon asks about the social network support. Gary and Mattias explain how the different addins can announce builds to Twitter, Slack, Gitter, Hipchat, etc. They describe how things like this that are useful but not essential are available as addins.
      • (46:28) Kevin asks if there are any addins that they see as missing. Gary says there aren’t at the moment. He’s got a generic build script for all of his addins, so adding to the script rolls out to all of his addins.   
      • (48:08) Jon asks about the discussion in their repo: How do we prevent Addin's becoming stale? Gary talks about the problem (a community member creates a useful addin, then stops maintaining the addin). Their plan is to set up a cake-contrib organization which can help with long-term ownership and support for community addins, allowing them to push NuGet updates as required if the original creator is no longer available or involved. The plan is to ask users to move their addins into the new cake-contrib organization and make the cake contrib user a co-maintainer. Note: Since this was recorded, they’ve set up this organization, as discussed in the wrap-up after the show.
    • Open Source Project Case Studies: NancyFx and IdentityServer
      • (53:14) Jon asks about some of the open source project build conversions to Cake, starting with NancyFx. Mattias explains some of the challenges, as well as the clear payoff: build script pull requests very soon after the conversion. Jon asks some nitpicky details about why the NancyFx build script script is spawning a process instead of using a Cake wrapper. Mattias explains the process by which build scripts are refined, and Gary points out the advantage of always being able to spawn a process if you need to do something Cake doesn’t yet support.
      • (57:43) Jon asks about the IdentityServer migration. Gary said the IdentityServer team had already done the .NET Core port running under PSake and just wanted to migrate to Cake to get cross-platform builds going. It’s also a lot simple of a build process because it’s doing less.
    • Assembly Merging
      • (58:50) Jon asks about ilmerge suport. Mattias says it’s pretty popular for tools, to allow you to distribute the tool as a single exe. Gary says he uses Fody Costura instead to handle assembly merging. Jon says he doesn’t think any assembly merging support is available yet for .NET Core. Gary agrees, and Mattias says that .NET Native support will be helpful when it arrives.
    • Future Plans and Getting In Touch
      • (1:00:08) Jon asks about future milestones and what’s on the horizon after the .NET Core release. Gary hints at some upcoming releases, which are covered in the post-show wrap-up.
      • (1:01:20) Gary talks about some upcoming speaking engagements, and Mattias mentions the Gitter chat, Twitter and GitHub issues as ways for users to ask any questions.
    • Post-Show Update
      • (1:03:10) Cake Contributions organization
        The Cake Contributions organization (cake-contrib on GitHub) is now in full motion. It's a common place where people can put their Cake addins & modules, if they want. They still maintain and for all purposes “own” the code, but they get access to resources like CI services, common build scripts, better exposure and the Cake core team can assist with merging pull requests, fixing issues and pushing to NuGet if addin grows stale or the author just too busy with life in general. It’s been well received and quite a few projects has moved over.
      • (1:03:52) 0.16 Release with .NET Core support
        So in addition to full .net Framework support, Cake now supports .NET Core (netstandard 1.6). There have been a few patch releases following this, so as of Oct 11 they're up to 0.16.2. Also, due to adding .NET Core support, it's now possible to debug a Cake file with Visual Studio Code. That's cool, because there's now cross platform debugging support for Cake. There's a blog post showing how to do that.
      • (1:04:25) Frosting
        Frosting is a stand-alone .NET Core runner and host for cake. Cake uses the Roslyn scripting API and provides a DSL for fetching dependencies like tools and addins. Frosting uses the .NET core SDK to handle dependencies, and your cake build script is really just a .NET Core console app. Both are running against the same Cake.Core and Cake.Common packages, they're just being hosted differently. This gives you some advantages - you've got full Visual Studio IntelliSense and tooling support, use the standard methods to package your build as a NuGet package, etc. Frosting also includes a dotnet CLI tool, so you can call dotnet cake.
      • (1:05:46) Cake for Yeoman support
        This adds a cake generator which makes it easy to bootstrap Cake, including a build script, bootstrapper scripts and config files in the current directory just by typing yo cake. There's also scaffolding support to create a new .NET Core project using Frosting with yeoman.
      • (1:06:18) Cake for Visual Studio
        This is the first release of the Cake for Visual Studio extension. It includes syntax highlighting, Task Runner Explorer integration, bootstrapper commands in the Build menu, project templates for Cake addins, Cake Addin unit test projects, and Cake Modules
    1 hr 7 min
  • Herding Code 218: React Native for Windows with Matt Podwysocki and Eric Rozell

    Episode 218

    The guys talk to Matt Podwysocki and Eric Rozell about React Native for Windows.

    Download / Listen: Herding Code 218: React Native for Windows with Matt Podwysocki and Eric Rozell

    Show Notes:
    • Hello and Quick Update on Reactive Extensions
      • (00:15) Matt Podwysocki and Eric Rozell both talk about their work on the Microsoft Partner Catalyst Team.
      • (01:00) Jon asks Matt what’s new with Reactive Extensions for JavaScript (RxJS). Matt talks about how RxJS took on a life of it’s own, as it’s being used by Netflix and Google with Angular 2. Matt’s still a core team member but there are plenty of other people contributing, which is nice. On Reactive Extensions for .NET (RX.NET), they’re working towards the .NET Foundation so it’ll get increased community support. Reactive Extensions are used a lot at Github and Facebook uses it for it’s Nuclide IDE.
    • React and React Native Overview
      • (02:55) Jon asks Matt to overview React. Matt explains how React handles views, components, and unidirectional databinding. Matt says it’s nice that it moves us towards the component model in the HTML world – moving away from the jQuery direct DOM manipulation approach to a more holistic approach.
      • (05:40) Jon says that he thinks that components introduce some problems of their own and asks Matt how React avoids that complexity. Matt says that’s why Flux and Redux were introduced.
      • (06:35) Jon asks what React Native is. Eric explains that React Native allows you to take the same concepts and tool used for building React web applications and use them to build native applications. It started at Facebook and has picked up a lot open source momentum.
    • How React Native compares to Cordova, Xamarin, etc.
      • (07:33) Jon says there’s always heated discussion when you talk about building native applications using web technology. What’s the overhead? Is this similar to Cordova? Eric explains the differences.
      • (08:50) Kevin asks you’d build a native application with React when there are lots of great native development tools out there. Eric talks about some of the advantages of building your application’s UI in JavaScript. Also, shipping updates as JavaScript files using CodePush can be a lot simpler than shipping a new binary for every update. Matt says you can also focus your development team on building the same application for all platforms using React skills rather than separate focus and development skills for each platform.
      • (11:22) Jon asks where React Native fits in on the scale from Cordova to more native tools like Xamarin.
    • Technical Implementation: How React Native creates Windows UWP applications
      • (13:35) Jon asks how UWP fits into React Native and how they implemented it technically. Eric says they took a look at how React Android was built and used that as a guide. He discusses how they handled the runtime, and how the application JavaScript is handled by Chakra.
      • (15:50) Jon asks about the application is hosted. Eric explains that they first create and navigate to a XAML page, then there’s an initial call to start the JavaScript engine with the application name, then sends back over the bridge a bunch of operations against the UI Manager. The UI Manager will then create subviews and construct the UI tree. In the React view, you have a virtual DOM that represents your UI; that gets translated to operations that are sent over the bridge to create a native UI tree.
      • (17:30) Jon asks about how elements are styled, since from his experience that’s all done using CSS styles. Eric says they use a subset of CSS called Flexbox using a library by a developer from Facebook. Matt says when they ported over the F8 app to Windows, they leveraged the same styles and platform specific style pattern that had been used for iOS and Android.
      • (19:23) Jon asks what was most difficult in moving the F8 application to Windows. Eric says the styling part was pretty easy. The hardest part was in handling platform specific support. Matt describes an issue with linear gradient support.
    • Nuts and bolts: Data, Extensibility, X-Plat Dev, Building and Publishing, Scaffolding, Tools, Testing, etc.
      • (21:00) Jon asks how they handle data flow. Eric says they use Relay, which runs on top of GraphQL. It all just moved directly over. Matt says that’s the beauty of this – if there’s something written directly in JavaScript it just works.
      • (21:55) Jon asks about any other platform specific issues. Eric says that it’s conceptually similar to Xamarin in that are you need to keep in mind that you’re writing to a native front end experience. He explains that it’s a happy medium between web front-end (Cordova) and native in that the simple UI components like text are easily shared, but you can target more platform specific controls as needed. He talks about some differences in things like menus, split views and carousels.
      • (23:50) Jon asks about the cross-platform development experience? Do you use a lot of emulators? Eric says he mostly works on Windows using the Visual Studio Android Emulator. He says when he’s adding a new feature to React Native for Windows, he’ll pop open the Android emulator and use the UI explorer. Matt says he also uses Android Studio and Xcode.
      • (25:44) Jon asks if there’s a long build step, or if there’s a quick feedback cycle. Eric says that React is built to make the developer experience really fast, and explains how hot module reloading makes it possible to see application code changes without reloading or even losing current running state. Matt says that you also get the full Chrome debugging experience.
      • (27:15) Jon asks how you package your application for publishing. Matt talks about the publishing steps.
      • (28:10) Jon asks about the rnpm (React Native Package Manager) command-line tool. Eric explains how rnpm helps you include dependencies in your React Native applications. It’s extensible, so Eric and Matt used that to implement the Windows scaffolding experience.
      • (29:52) Jon says that in the scaffolded application he got when he ran rnpm, there are OS specific files for each view and asks if it’s possible to use a consolidated view for all platforms when appropriate. Eric says you’ll generally need a platform-specific entry point for each OS, but once you get past that you can do quite a bit with the core React Native elements.
      • (31:15) Jon asks if it’s possible to create additional platform-specific controls. Matt talks about support for native modules (e.g. dialogs) and  native components (e.g. ImageView).
      • (33:20) Jon asks how he actually runs the application as he’s developing it. Matt says they just read in from the command-line to start the packager and build and run the application using Visual Studio. The docs currently say Visual Studio is required, although that isn’t necessarily the case if you kick off your build using msbuild.
      • (34:30) Jon asks if it could be possible to create a React Native Windows application on Mac or Linux, or if you’ll always need Windows due to the SDK. Matt says you’ll always need the SDKs, but Eric points out that there is a web-based React Native Playground (rnplay.org), and they’d like to reach out to them to get Windows support added.
      • (35:33) Jon asks if anyone is using it beyond the F8 application. Matt says not yet, they’re still working on the dev experience before they start heavily onboarding people. Jon asks who’s using React Native in general and Matt points out the React Native Showcase.
      • (36:50) Since part of the idea of UWP is that it runs on multiple Windows platforms, Jon asks if they’ve done anything with that. Eric talks about how part of their F8 demonstration included showing the application on Xbox, mobile devices with Continuum and desktop. They’re interested in looking into HoloLens and IoT as well.
      • (38:00) Jon asks if it’d be possible to leverage React web code in a React Native application. Eric says that at F8 he saw a talk on the possibility of bringing React Native to web. There are definite sharing points in code between web applications and React Native apps, although you should expect to write native views, and points out that there is work going on to bring React Native to Tizen and macOS.
      • (39:35) Jon asks if React Native for Windows is a science project or an ongoing project. Eric and Matt say they’ll be watching for community interest. They expect to keep up with React Native releases, and are hoping for community contributions. Eric says there are hundreds of React Native community modules, and they can’t scale to supporting all of them without some community involvement. 
      • (41:15) Jon asks about Visual Studio Code support. Matt says that they’re working closely with the VS Code Team. They’ve already got a plugin for React Native that supports deployment to iOS and Android, and Matt’s hoping to get UWP deployment added to that list. Eric points out that you can also set VS Code as your remote debugger, so you can set breakpoints, step through your running code, view stack traces, etc.
      • (42:18) Kevin asks about the testing story – can you write unit tests and integration tests for React Native? Eric says it’s similar to building large cross-stack applications with tests for services and clients - where you write integration tests between your JavaScript libraries and native code, unit tests for native modules, and tools like Jest for your pure JavaScript components.
    • Getting Started
      • (43:44) Jon asks for other resources for people getting started. Eric points to the excellent documentation for React and React Native. For React Native for Windows specific docs, there’s additional documentation in that repo. Matt recommends following the @reactwindows Twitter account.

    Top Links:

    • Announcement: React Native on the Universal Windows Platform
    • Repo: https://github.com/ReactWindows/react-native-windows
    47 min
  • Herding Code 217: Nick Craver on Stack Overflow Engineering

    Episode 217

    The guys talk to Nick Craver about all the magic behind the scenes at StackExchange global headquarters.
    Download / Listen: Herding Code 217: Nick Craver on Stack Overflow Engineering
    Show Notes:
    • Hello
      • (00:15) Nick explains his job: software development, sysadmin (site reliablity engineer), and sometime DBA. Just not devops.
      • (01:30) Jon introduces Nick's recent blog post series and the Trello board where people can recommend and vote on new topics:
        • Stack Overflow: A Technical Deconstruction
        • #1: Stack Overflow: The Architecture - 2016 Edition
        • #2: Stack Overflow: The Hardware - 2016 Edition
        • #3: Stack Overflow: How We Do Deployment - 2016 Edition
        • Trello board: Blog post queue for Stack Overflow topics
        • Architecture / Network
          • (01:58) Jon notes that they're using Windows and CentOS and asks why CentOS as opposed to other Linux flavors. Nick says they tried Ubuntu first, but it's more tuned for clients. CentOS is a variant of Red Hat Enterprise Linux, so all the packages work.
          • (02:53) Nick says that they use whatever's the best tool for each job - factoring in the costs of each new tool (training, migrating, supporting, vendor overhead, etc.). They run Elastic Search, Redis, HAProxy and Logstash.
          • (03:50) Jon asks about how they're using protobuf to serialize the information they're persisting Redis. Nick talks about some specifics, including the different levels of caching that they're doing. They're using pub / sub with Redis via websockets. They're not clustering, partly because there's one Redis cache per SE site. He and Jon discuss how multitenant scenarios often require custom implementations.
          • (07:20) Jon asks about how they're using websockets. Nick says that they're used in a lot of places for optional updates. Their problems come from running on very few servers - they end up with huge connection tables per server.
          • (08:23) Jon asks about one of their Redis instances that's handling machine learning with Providence. Nick says they log some metadata and performance information about about every single request. Providence is a system their data team wrote that analyzes the data, figures out locations, suggested user tags, etc. As a user, you have control over the personalization and can download the data or disable recommendations if you want. There's also a mobile feed, for mobile apps as well. There are 40k ops/sec all day long against Redis. Scott K asks if he can manipulate his feed.
          • (11:51) Scott K asks about the L1 and L2 cache that Nick's talked about. Nick clarifies that he's been referring to HTTP caching on the web server and Redis caching.
          • Tag Engine
            • (12:36) Nick talks about the tag engine and explains how it's a lot more complex than people think since it handles some complex queries, needs to sort the results, has minimum question score filters, etc. Every two minutes, the tag engine refreshes all the deltas based on row versions in the SQL Server database.  Their previous implementations were saturating the CPU L3 cache. They wrote some affinity code to split things out so Providence and Stack Server run on different processors. Marc Gravell is now working on getting the tag engine working on GPU's (demo code). Nick said that the difference between a $500 to a $5000 card is only about a 40% improvement. They're really excited about upcoming graphics cards with 8GB memory, since that will allow them to fit the entire tag engine in memory on the graphics card; with two cards they'll be able to run active-active and active-passive configurations (for live upgrades) for only about $600 per card. Jon says he saw a question on Hacker News asking if it could just be handled by a bitmap index, but Nick explains how more complicated queries and sorting make that not a good solution. Jon asks about the GPGPU split between C# and C++ code. Nick explains how it's almost all handled via C# code, but it's missing a few newer CUDA commands.
            • (19:48) Jon asks about the Elastic Search implementation. Elastic Search doesn't really support types, it has field groupings, which makes the upgrade more difficult. Nick explains that things are pretty vanilla now, but they'd like to make some customizations to support nested search results when time permits.
            • Data and SQL Server
              • (21:03) Jon asks about their SQL Server implementation. Nick talks about the clustering setup.
              • (21:49) Jon asks what version of SQL Server they're using. Nick says they're currently running the latest version of SQL Server 2014 and will move to SQL Server 2016 as soon as it releases. [Note: SQL Server 2016 has since released and they've upgraded.]
              • (22:11) Nick talks about some of the top reasons they're looking forward to SQL Server 2016: string_split and JSON parsing. These are both useful for queries that take a list as a parameter. Jon reminisces about a time long ago when he used XML to pass lists to SQL queries.
              • (23:32) Kevin asks if they're able to do a piecemeal migration without downtime. Nick explains how they do upgrades using replicas. They can test on other replicas, then fail over to them, or roll back to the previous master. They hate Windows clustering, and Windows Server 2016 and SQL Server 2016 should soon support distributed affinity groups which would allow them to do simple affinity group based upgrades.
              • (26:15) Jon asks about SQL Server on Linux. Nick says he can't really talk about it.
              • (26:35) Jon asks if they use SQL Server Hekaton / In-Memory OLTP. Nick says they don't, they run enough memory in their database servers that it's not needed. Nick says it's more for high-frequency no-lock access.
              • (27:27) Jon is a little horrified that they don't use stored procedures and do all database queries using inline SQL. Nick explains that it's much simpler to modify queries and app code that consumes the query together rather than managing separate deployments. There's no performance benefit to using stored procedures over inline queries.
              • (29:27) Jon asks about how they handle migrations. Nick explains how changing tables is handled via a migration file and a ten line bot.
              • Source Control, Localization, Build
                • (30:47) Jon asks about their use of GitLab. Nick says it works okay, but they're testing GitHub Enterprise internally due to performance. GitHub is significantly faster, search works a lot better (due to using Postgres search rather than Elastic Search), and there are some nice new features in GitHub like squashing commits.
                • (32:23) Jon asks about the localization features and is educated about the ja, ru, pt and es versions of stackoverflow. Nick explains some of the different localization issues that you run into in localization. Most localization solution work by string replacement, which requires string allocations. That doesn't work at scale. They've written a system called Moonspeak which uses Roslyn to precompile view. This allows a direct response.write of the localized string via switch statements, which is a lot more efficient. They haven't had time to open source it yet, but would like to.
                • (36:36) Jon asks about their build process using MSBuild. They use it mostly because it's what the tooling uses. They could customize it more with PowerShell, but that would tie them more to TeamCity and Nick's not sure there's a benefit to making that move. Nick's waiting to see where csproj is going - he's got some big doubts it'll be as terse as project.json, but he's interested to see. Nick says historically MSBuild has been optimized for three-way merge generation; Jon says that was technically Visual Studio's fault since MSBuild actually has had glob support for a while.
                • Upcoming Technologies, Visual Studio
                  • (40:35) Nick complains about how slow Visual Studio is to reload projects. Their developers have scripts that just kill and restart Visual Studio, because that's faster than handling project reloads.
                  • (41:16) Jon asks if Nick's played with Visual Studio "15". Nick wonders about the technology used in the installer. He says it's generally good, but they're running into some issues with solution files changing when moving between versions.
                  • (42:33) Nick says that they generally don't ever use File / New, they copy an existing project and rename things. There's a discussion about whether it's possible to customize project templates. Jon says you can export a project as a solution template; K Scott mentions that SideWaffle has some capabilities there, too, but there was some "wonkiness". And what's the deal with GUIDs in project and solution files?
                  • (46:09) Scott K mentions a command line base project generator that he started on years ago called ProjectStarter. He wishes that it was possible to configure Visual Studio to define a custom build tool rather than assuming everything's in csproj. He gets that Visual Studio features like IntelliSense depend on controlling the build, but doesn't like that Visual Studio has to "know everything about everything".
                  • (48:55) Jon says he sees two ways that cross-platform can work: either make the frameworks able to work without the tools knowing and controlling everything, or updating Visual Studio Code so it's able to know and control everything. He hopes it's the first way.
                  • (49:50) Nick complains about how sometimes in-memory builds don't reflect changes, or csproj doesn't save before a build. He'd like everything to save before builds. Scott K calls out the Save All The Time extension for Visual Studio that Paul Betts made.
                  • (50:40) Jon asks Nick if they've looked at ASP.NET Core. Nick says that they'll mostly be starting with their internal tools. They have several libs that they'll need to port, and they've got some difficult problems with libraries like MiniProfiler that need to support both .NET 4.x and .NET Core because the underlying APIs have some significant differences. You can't just multi-target code that targets things like HttpContext. Other libraries like dapper and stackexchange-redis haven't been as bad, and they've been working on them because lots of other developers are depending on them.
                  • (55:03) Jon calls out some of Nick's recent C# 6 tweets. Nick says he likes null coalescing and ternaries - they see more terse code as a lot more readable, but it of course varies by team. Roslyn has been really big for them, things like Moonspeak rely on it.
                  • Questions from Twitter
                    • (56:23) Matt Warren asks "What performance issues have you had the most fun finding and fixing." Nick mentions the tag engine and a fun debugging issue they ran into where the TimeSpan int constructor uses Ticks rather than seconds or milliseconds, so their cache code was only caching values for tiny fractions of a second rather than thirty seconds. They find out so many issues using MiniProfiler; he wishes more developers would use MiniProfiler (or another tool like Glimpse) in their applications. They run MiniProfiler for every single request on StackOverflow and the overhead is minimal - if they can do it, you can do it.
                    • (58:17) Matt Warren asks "What the craziest thing they've done to increase performance." Nick talks about the IL related work they've done - sometimes instead of conditional code, it's faster to just swap out the method body. They're pragmatic, they only do this for extreme cases like things that run for every request and have real performance implications. What's the trick for StackOverflow? Keep it simple.
                    • 1 hr 2 min
                    • Herding Code 216: Bash on Windows, Angular 2, Surface Book, Kindle Oasis, Windows Phone

                      Episode 216

                      Kevin, K Scott and Jon talk about Bash on Windows, Angular 2, React, new tech devices, and whether Windows Phone is alive, dead, or undead.

                      Download / Listen: Herding Code 216: Bash on Windows, Angular 2, Surface Book, Kindle Oasis, Windows Phone

                      Show Notes:
                        • Bash on Windows and Windows Insider stuff
                          • (00:44) Jon mentions the Bash on Windows announcement at Build and asks if Kevin or K Scott have played with it. This devolves into a discussion of Windows Insider previews. Jon likes it and talks about the steps for enabling Windows Insider preview builds. K Scott has been scared to try it, but it sounds like he's convinced. Kevin is put off by the Insider term - Windows Insider, Visual Studio Code Insider previews, etc. K Scott adds "Windows Insider" it to his e-mail signature.
                          • (05:15) Jon talks about the steps for enabling Bash on Windows. or Windows Subsystem for Linux (Beta), to use the official terminology.
                          • (07:08) Feel the excitement of listening to someone type commands into a console window as Kevin asks questions about what's installed and Jon tries to apt-get it all. K Scott and Kevin wonder about how things like filesystem and processes work, and Jon tries to make up answers.
                          • (09:50) Kevin says it feels like an admission of defeat to add *nix support to Windows. Jon says it feels practical to him - developers are building for multiple operating systems (especially including mobile), so it's nice to have it supported.
                          • (11:19) Kevin's ready for Cygwin to die in a fire, and Jon's excited about ssh working less horribly on Windows. Kevin says the race is on to get Wine working on Bash on Windows.
                        • Angular 2 and React
                          • (12:55) Jon worked on a hands on lab for Build that had master-details using Angular 2 and ASP.NET Core. He said Angular 2 seemed a lot simpler than Angular 1 now. K Scott said the component model is simpler, but he's seeing some resistance to the ECMAScript / TypeScript updates, new binding syntax, etc. The Angular 1.5 release also includes a component model that's a much easier programming model. It almost feels like some older Microsoft component-based programming frameworks going back to Visual Basic 6: you're working with components that have simple properties and events. The guys speculate on how soon someone will build the big visual editor for Angular 2.
                          • (16:21) Kevin says TypeScript seems like a barrier. K Scott says it's not strictly required. He rejected TypeScript for a while, but when he was working with Angular 2 and tools and editors supported it he decided he liked it. Jon and K Scott talk about how a lot of things that throw people off about TypeScript are really just modern JavaScript syntax.
                          • (18:50) The guys discuss how Angular 2 and React mindshare will play out. Jon likes React as long as he never views source. Jon thinks the unidirectional flow is really simple, and Kevin agrees - after years of lower level Backbone, the simpler flow in React saves some mental energy.
                          • (20:29) Jon mentions that React Native recently came to Windows, too.
                        • Devices: Surface Book, Kindle and big batteries
                          • (20:56) K Scott got a new Surface Book (after waiting to make sure nothing new was coming out at Build). He says it's the best piece of PC hardware he's bought in years - the build quality is good, the keyboard is good, he gives the trackpad of 9 out of 10. He says that the detachable tablet is a bit large as a tablet, so he's using an older Surface for reading.
                          • (23:58) Jon jokes that K Scott's not likely to buy a Kindle and says he gradually stopped using his Kindle when he moved to Audiobooks, and kind of associates reading with work now. Kevin says that's sad. They talk about Kindles for kids' books.
                          • (27:07) Kevin says the two big things he picked up for the new Kindle are physical page buttons and a three month battery. He says the main thing he likes about Kindle for both him and his kids (as opposed to a tablet) is that it forces you to read instead of getting distracted. The guys decide that tripling the life of an already one month battery isn't a huge win.
                          • (29:20) Jon says he recently bought a portable battery that can recharge his laptop, which is handy for long flights. (note: he said it was iPad size, it's a lot smaller but is 1.2 pounds)
                          • (31:15) Jon asks Kevin about new Apple hardware. Kevin says the iPad Pro screen is apparently astounding - he's expecting them to be amazing in a few generations. Same for Apple Watch - he doesn't have one yet, he's waiting for version two. Jon says the improvement from Microsoft Band 1 to Band 2 was pretty nice, especially in the industrial design.
                        • Lightning Round: Will Windows Phone be dead in one year?
                          • (33:40) K Scott asks if Windows Phone will be dead in a year. Jon hopes not, as he just bought a Lumia 950 XL. He'd had a budget Blu Windows Phone since September, and the Windows 10 Insider builds were nice, but the camera wasn't very good. He really likes Windows 10 as a phone operating system and thinks it's sad that so few people will actually see it.
                          • (35:55) K Scott got a Lumia 950 XL in January (when he dropped something on his previous phone). He got the docking station, too, and said it worked surprisingly well. The guys discuss how useful docking a phone is; Jon postulates that it could be useful for someone who does everything on their phone and occasionally needs to write a long email or edit their resume - especially if it can be hooked up to a TV.
                          • (38:30) Kevin says that Windows Phone isn't dead. it's undead. It will linger on in a zombie-like state indefinitely.
                        • Scott Koon sends us out with a request for further information by e-mail.

                      Links:

                      • Windows Insider site
                      • How to install Bash on Windows support
                      • Build 2016 Angular 2 / ASP.NET Core Code Lab
                      • Angular 1.5 overview, including component support
                      • React Native on the Universal Windows Platform
                      • React Windows on GitHub
                      • The portable battery Jon got: RAVPower 23000mAh Portable Charger Power Bank External Battery Pack
                      42 min
                    • Herding Code 215: Jon McCoy on .NET Security and Defensive Patterns

                      Episode 215

                      At NDC Oslo, Jon talked to Jon McCoy about .NET security and defensive patterns for building enterprise applications.

                      Download / Listen: Herding Code 215: Jon McCoy on .NET Security and Defensive Patterns

                      Show Notes:
                        • Security Patterns
                          • (0:15) Jon Galloway (henceforth in this post JG) mentions the last time Jon McCoy was on the show and asks him about his talks this time around. This time, he and Topher Timzen did a hands on attack class followed by talks on defensive patterns for enterprise applications (video: Hacking .NET(C#) Application: Building and Breaking Layered Defense).
                          • (00:54) JG asks Jon if he showed off any scary hacks. Jon describes an attack in which they made an executable editable both on disk and in memory, then edited both the IL and assembly code to do things like inject direct database attacks (bypassing . He describes how that could be defended using enterprise defensible architecture by talking to databases through services which can implement security layers. The goal is to prevent an attack at one layer from moving through the rest of the enterprise.
                          • (1:59) JG says that often hears that major hacks occur by web application attacks that are then escalated to database attacks, often through password reuse. Jon says that's true, since web applications are often deployed to the same server as a database or authentication server. He recommends using a service that's locked to a single port, with security unit tests.
                          • (2:43) JG asks which patterns he's describing are unique to .NET development. Jon says that he's emphasizing patterns that are easy on .NET - for example,  REST services are easy to implement on .NET as compared to C++. He's advocating architectural changes that are relatively easy to implement in .NET applications provided you start with them early on (rather than trying to retrofit security later).
                        • A specific example: Protecting a medical record
                          • (3:33) JG asks for some specific examples. Jon says they talked about security unit tests and user stories and gives and example from his talk about a medical record that's being sent through an enterprise securely. To do that, you'd need to encrypt it on an edge node, so the web server and database don't have decrypted data or decryption keys. Instead, you use a key server on a segmented network. Because of this, at no point could a sysadmin have gotten access to the record because it was encrypted at all steps.
                        • "Above Admin"
                          • (4:49) JG points out that Jon is talking about preventing access to sensitive data by sysadmins. Jon says that you should consider your attackers to have more power than a sysadmin - they refer to attacker privileges as "above admin" because they've taken over your AD infrastructure, passwords on routers, dropping new firmware, etc.
                          • (5:50) JG refers to James Mickens' keynote from the previous night (Herding Code interview here) and asks if there's any hope. Jon says you need to plan, you need to minimize pivot points to prevent an attacker from moving between servers. He discusses potentially using static HTML that calls into secured REST services rather than web applications with direct database access, because building security around a REST service is much easier than securing a web application and web server.
                        • Mobile and desktop application security
                          • (9:03) JG asks about mobile application security, since he frequently hears about mobile apps that are supported by unsecured (or very poorly secured) backend services. Jon refers to Amazon and Twitter as examples of companies with published patterns for secured backend services.
                          • (9:52) JG asks if for some tips on how different systems on the network should be secured, referring to desktop applications. Jon says that each system should defend itself from the other systems, so in this case the other systems should assume that the desktops could be compromised, the desktop applications should assumed the database can be compromised, there needs to be thought about defending the outgoing APIs, etc. There needs to be a plan for how to take things down and respond.
                        • What do you wish you'd done?
                          • (10:35) As a thought exercise, assume that your database or web server has been compromised: what do you wish you'd have done? Do that for different pieces of your application architecture. Jon says that you can make it a fun exercise, pit dev teams against each other, etc. Security should be fun and easy if you're doing it right.
                        • Wave 3D: A 3D operating system front end
                          • (12:07) JG asks Jon about the 3D operating system he'd mentioned before the call. Jon describes Wave 3D, a cross platform front-end to multiple operating systems (Windows, Mac, Linux, Android) that gives the same exact experience for all backend systems. It can be connected to Amazon, Google and Dropbox document storage.
                          • (12:56) Jon says it's running on Mono and Unity and says it compares pretty well with 3D operating systems in the movies, and they're looking at launching it via Kickstarter. It provides a simple, install-free environment with a document viewer, movie player and more on every platform instantly.

                      Note: This is our last podcast from NDC 2015.

                      15 min
                    • Herding Code 214: Todd Gardner on Tracking JavaScript Errors with {Track:js}

                      Episode 214

                      At NDC Oslo, Jon talked to Todd Gardner about {Track:js} and some of the difficulties in diagnosing JavaScript errors.

                      Download / Listen: Herding Code 214: Todd Gardner on Tracking JavaScript Errors with {Track:js}

                      Show Notes:
                        • What's TrackJS?
                          • (0:20) Jon asks Todd to explain TrackJS. Todd said it's something he built to help with difficult JavaScript debugging issues a few years ago. JavaScript applications were difficult to debug because it was hard to define the different failure states of his applications. Over time, they worked to add in diagnostic state (e.g. shopping, checkout, etc.). Over time, they added in network state and user state. Over time they came up with telemetry, which defines states and state changes, which can be a lot more helpful than a stacktrace in diagnosing and fixing an error.
                        • Why context is more important than stack traces
                          • (2:34) Jon agrees, saying that it's really frustrating to try to reproduce an error as a developer, and context is really important. Todd says that stack traces are limited because applications are so asynchronous in nature now - an error may be from a timeout which was a response to an Ajax event which was a response to a user click.
                          • (3:44) Todd says they also track a lot of general information like browser, operating system, and things like date and time information. He once spent a really long time on an error one user was experiencing on a calendaring application because they were stuck in the '70's (literally).
                        • How TrackJS gathers and exposes data
                          • (4:34) Jon asks how developers integrate TrackJS into their applications. Todd says it's a simple JavaScript include.
                          • (4:56) Jon ask more about how he gets access to the reported information when he's troubleshooting an error. Todd says there are three problems to solve.
                            • The first problem to solve is alerting: you get daily rollups on top errors and trends.
                            • Second, when you log in, you need to see which errors are which important (filtering out background noise like unsupported useragents, strange Chrome extensions that modify the DOM, etc.). You need to know which errors impact the most people, which cause users to leave, and which are top browser-specific errors.
                            • Third, you need to need to be able to dig into and diagnose a specific error.
                          • (6:45) Jon asks more about how he'd diagnose a specific error. Todd describes the contextual information TrackJS provides for an error (including things like browser, screen size, user information). For example, one issue they diagnosed recently only occurred at a specific screen size due to a responsive design which modified the page so certain elements were only displayed in a range of screen widths.
                        • Common problem: Partial Load Failure
                          • (8:14) Jon asks Todd what running the service has shown him are common problems that developers aren't considering in building JavaScript apps. Todd says it's partial loading failure - cases where most of the JavaScript assets on a page load, but just one fails. In one instance, they saw everything on a page but the Stripe library load, which caused payment data to be insecurely posted.
                          • (10:35) Jon asks about the best way to test for and handle partial load failures.
                        • Common problem: Same Origin Policy
                          • (11:36) Todd explains same origin policy restrictions in loading loading libraries from other domains, and using cross-origin headers to handle that.
                        • Business model and subscription overview
                          • (12:30) Jon asks about the business model for TrackJS. Todd explains that they offer free service to charitable and open source projects. For paid accounts, they bill based on page views rather than error count because he feels billing based on errors creates counter-incentives for users.
                      15 min
                    • Herding Code 213: Sean Trelford on Composing 3D Objects with F# and OpenGL

                      Episode 213

                      At NDC Oslo, Sean Trelford did a lightning talk on composing 3D objects using F# and OpenGL. Oh, and he's 8 years old. Sean (and his dad, Phil) talk to Jon about learning coding with Small Basic and F#, and how it's fun to learn coding by building video games.

                      Download / Listen: Herding Code 213: Sean Trelford on Composing 3D Objects with F# and OpenGL

                      Show Notes:
                        • Hello
                          • (0:34) Sean is eight years old, but that didn't keep him from giving an NDC talk on composing 3D Objects with F# and OpenGL.
                        • Sean's Talk
                          • (1:13) Jon asks Sean whether he showed slides or did live coding.
                          • (1:39) Sean made an army from one 3D man model.
                          • (2:09) Jon asks how difficult it is to call OpenGL from F#. Sean's dad, Phil, explains how Sean used a domain specific language - about 50 lines of code. You can use it to create shapes like cubes and spheres, color them, transform them, etc.
                          • (3:30) Phil explains how Sean used the F# REPL to create and modify 3D objects interactively, then used functional composition to put the parts (head, arms, legs) together to build the army man, the used functional composition and aggregation to create the army.
                        • Learning Small Basic and F#
                          • (4:37) Jon asks Sean if F# is his first computer language; Sean says he started with Small Basic. Jon asks if he found it difficult to move from Small Basic to F#, Sean and Phil explain how F# is actually a pretty natural step from Small Basic. Small Basic has no functions with parameters, so when they had a project that ran into that limitation they just moved the code from Small Basic to F# and used the Small Basic library to construct 2D shapes (leveraging WPF).
                        • Fun Basic
                          • (7:02) Phil said that last year he and Sean rewrote Small Basic as Fun Basic (available free in the Windows Store). It's an IDE with IntelliSense, code completion, etc. It's got backward compatibility with Small Basic, but includes function with arguments, pattern matching, etc.
                        • Learning programming with video games
                          • (8:02) Jon asks about what next for Sean. Recently they went to a game jam and made a platform game. Phil says the REPL environment is huge for game building.
                          • (9:04) Jon mentions the IoT lab at NDC and asks Sean if he's done anything with IoT; Sean says no. Jon says it seems like people try to get kids interested in coding with IoT, but maybe games are more fun to get started. Phil says that Sean's got a Minecraft channel on YouTube. Jon asks Sean about making videos about making games.
                          • (10:38) Phil says he got started on computers with games - he made his first game when he was 11, but his first commercial game was when he was 14 - Flint Eastwood.
                          • (11:44) Jon asks Sean if this is his first time in Olso, and his first conference. It's his first time in Oslo, and his first "speaking" oriented conference.
                          • 11:28 Jon asks if they've got any last messages, maybe to encourage kids to get started with coding. Phil recommends the Hour of Code, as well as other resources online. Phil says his older son got into programming by adding levels to games, then writing scripts for AI's, etc. Phil says that's how he got started - looking at other peoples' code and going from there. Don't necessarily get hung up on fancy coding concepts, just script something and have fun!
                      15 min
                    • Herding Code 212: Steve Sanderson on Web Development in 2020

                      Episode 212

                      While at NDC Oslo, Jon talked Steve Sanderson about Web Development in 2020 - future JavaScript features, adventurous optimizations, and constraint-based styling.

                      This week we've also got a full transcript (thanks to Damian Guard!).

                      Download / Listen: Herding Code 212: Steve Sanderson on Web Development in 2020

                      Show Notes:
                        • Hello
                          • (0:34) Steve summarizes his NDC talk on the future of Javascript
                        • ES 2015 & 2016
                          • (0:52) Jon asks if ES 2015 and 2016 is the new name for ES 6 and 7
                          • Steve explains how the standard body are going with yearly updates
                          • (1:25) Jon asks what some of the new features are
                          • Steve talks about how it looks so different and how the import and class features help
                        • Polyfils & transpilers
                          • (2:14) Jon asks if he can use those features today
                          • Steve explains the polyfills and transpiler options
                          • (2:46) Steve talks about the Babel transpiler open source project and why people should write ES 2015 code today
                        • What about Typescript?
                          • (3:19) Jon asks if Typescript is still an option
                          • Steve explains how Typescript is a transpiler and provides some similar features as well as strong typing
                        • Web Components
                          • (4:05) Steve talks about web components, how they're not implemented in browsers today and the polymer.js project that can bridge that gap
                          • Steve talks about scoping, the Shadow DOM and how components are encapsulated from the rest of the page
                        • Async & await
                          • (06:04) Steve talks about his interest in the upcoming async/await support and how it goes further than promises in order to clarify the flow of success and errors
                          • Steve mentions that Babel can polyfill async/await support today
                        • Generators
                          • (7:30) Jon asks for an explanation of generators
                          • Steve compares generators with the yield function of C# and explains how the function is marked with an asterisk to signal it is a generator
                          • (8:08) Jon asks if this is the end of promises
                          • Steve explains that generators are built on top of promises and how they can chain waiting for multiple ones sequentially
                        • Emscripten
                          • (8:53) Jon asks about Emscripten
                          • Steve talks about the Emscripten bytecode format and how it can make Javascript faster as well as providing the foundation for new web languages
                          • Steve talks about the LLVM bytecode converter and how it only really works with languages like C and C++ that don't have a runtime like Swift, Rust, Go etc.
                          • (12:06) Jon asks about the demo compiling SQLite
                          • Seve talks about why SQLite works when others don't.
                        • Shared Array Buffer
                          • (12:48) Jon asks about the Shared Array Buffer
                          • Steve talks about how Javascript took an isolated approach to parallelism and how Shared Array Buffer allows the breaking of this boundary to enable some high-performance parallelism
                          • (14:05) Jon mentions how the ASP.NET 5 team are doing similar things for performance
                        • Grid Style Sheets
                          • (15:00) Jon asks about Grid Style Sheets
                          • Steve points out the "Grid" in Grid Style Sheets refers to the company behind it and not the approach which is a rule, constraint and solver layout system like Auto Layout in iOS
                          • (17:14) Jon points out it's Javascript and so people can use it today
                          • Steve talks about where it might go and explains how variables are much different to typical variables
                          • (18:34) Jon asks about the animations in the demo and Steve reveals it's a CSS transition
                          • Jon and Steve discuss how GSS and CSS might not mix well together
                          • Steve warns GSS is really cutting-edge
                          • Jon asks about debugging constraint rules and Steve agrees this is a challenge right now
                        • Goodbye
                          • (20:30) Jon says goodbye as Steve heads off for his plane.

                       

                      Transcript

                      [00:00:17]

                      Jon: Hello and welcome to Herding Code. I am sitting here at NDC speaking with Steven Sanderson or Steve Sanderson, sor ry.

                      Steve: Either's fine with me.

                      Jon: Either's fine, great. I caught the tail end of your talk yesterday. It was fascinating, you were talking about future Javascript features or browsers features.

                      Steve: That is right - I was trying to predict the future of client-side web application development. I don't know how accurate my predictions will be but some of them are fairly safe bets like we're going to start using ES 2015/2016 stuff pretty soon and then there's a whole raft of other features coming towards us.

                      [00:00:52]

                      Jon: Ok. So that was actually the first time I've seen ES 2015 and 2016. I think I've heard in the past ES 6 and 7.

                      Steve: Yeah, it's just a bit of new marketing really. The folks behind the standards there have basically committed to bringing out a new update every year. So we've got this annual cycle rather than up until know it's been every five years or every ten years in one case. So given that it's going to be annual it it sort of makes sense to give the name a year.

                      Jon: Right, yeah, definitely. Kind of like Visual Studio has done.

                      Steve: Exactly.

                      [00:01:25]

                      Jon: So what are some great features there in 2015/2016?

                      Steve: Yeah OK so ES 2015 was previously known as ES 6 has got to be the biggest set of changes that have come in Javascript since it was first introduced. If you look at something that's written with ES 2015 syntax - if you've never seen that syntax before you would probably not even guess it was Javascript. It looks really different. So you've got import statements - which give you a clean alternative to having loads of script tags. You've got actual classes which are the same at runtime as regular Javascript prototypical inheritance but gives you a nice syntax for declaring that you can remember.

                      Jon: We're finally taking advantage - class has been a reserved word in Javascript forever, right?

                      Steve: Oh, yeah, I think that might have been the case.

                      [00:02:14]

                      Jon: Can I start using those features? I know there's some poly-fills and transpilers or whatever.

                      Steve: Yeah, I'd say if you want to start using ES 2015 today it's really not as difficult as you might think. Assuming you want to target all browsers since, let's say IE 9, then you can basically use almost all of the ES 2015 features. You just need to use a transpiler.

                      The one that everybody seems to be most into at the moment is called Babel (BAY-bel) or some people call it Babel (BAB-bel). I don't know what the right pronunciation is.

                      Anyway it's an open source project that will take your ES 2015 code and produce an ES 5 equivalent of it. It's really fast, easy-to-setup and to be honest with you I can't think of any good reason not to be doing that. If you're not doing that you're sort of just keeping yourself in the past really because this thing is a fully ratified standard now. We know this thing is definitely landing - all the browser makers are committed to bring this in. There isn't really a good reason to not be using this today.

                      [00:03:19]

                      Jon: I've looked at using Typescript for that kind of class support in the past. Is there any kind of downside to continuing with that or are they similar?

                      Steve: No there's certainly no downside to that. If you're getting benefit out of Typescript then absolutely use it. Typescript in a way is a transpiler because it is taking some syntax that is a lot like ES 2015 - it just adds some additional features on top of that to do with strong typing. So if you're getting some benefit out of that strong typing then yeah definitely use Typescript. On the other hand if you don't want to use the strong typing then you'll find that something like Babel will do the compilation, like an order of magnitude, faster.

                      [00:03:54]

                      Jon: So that's kind of a no-brainer as far as a future to dig into. What else did you talk about that's more forward looking?

                      Steve: I was also looking at web components which is sort of all tied up with ES 2015. It's kind of a part of this package of things arriving at the moment but unlike that making that work across all browsers is not quite as guaranteed today. So the best known polyfill for web components is polymer.js but in terms of IE that's only IE 11 onwards. But it's a really good feature. Web components is a way of combining together a bunch of scripts and markup and styles to make a reusable package of UI features that you can put in your application and you can assign them to a custom element name so you can have something like sales-chart as an element and when that goes into your application your custom sales chart appears and it has all the behaviors your want. And it can be, to some extend, encapsulated from the rest of your page as well. So this is a feature called Shadow DOM and it's a way of telling the browser that a certain element should be abstracted away as far the page that contains it is concerned. Code running outside it won't see the elements inside. If, for example, you were using jQuery and you were using a selector and it happened to match something inside that component it won't actually match it because it's considered to be like a separate document. So you don't get accidental leakages of behavior into your component or out of it - it just makes reuse easier.

                      Jon: And the same for stylesheets as well?

                      Steve: Yeah, that's right. If you bring a stylesheet into your web component and it's certainly going to apply inside that Shadow DOM root it won't apply outside it.

                      [00:05:31]

                      Jon: And I believe you mentioned polymer.js is the polyfill for that?

                      Steve: Yeah, it's the best known polyfill but Polymer is only the way of doing something like web components. If you're going to use something like React or Knockout or Angular any of those kinds of things they've all got their own kind of concept of reusable components with custom elements so if you're using any of those things you probably can pretty much do this anyway in a fairly cross-browser way.

                      [00:05:56]

                      Jon: Right. Ok. So then what other features did you cover?

                      Steve: I was interested in the upcoming await/async support. 

                      Jon: That's always been kind of a challenge in Javascript.

                      Steve: It so is isn't it. Yeah. So Javascript for the long time was plagued by this everyone's got a slightly different way of doing callbacks thing which is quite troublesome and then it all got formalized around promises. Promises is a fantastically elegant concept that gives you a really clear way of working with callbacks and chaining them and combining them and so on. So promises are brilliant and they're now a native browser feature but there's still a lot of opportunity to get confused with promises because there's so many way you can chain them together and not necessarily even understand how success and error values are flowing through that chain that you can still get things wrong by mistake. I find this happens all the time in my work. Not me, of course, but other programmers get very confused.

                      Jon: Right.

                      [00:06:55]

                      Steve: I might do sometimes. Whereas as any C# programmer knows with await and async keywords you can write your code as if it was synchronous.

                      Jon: It's so obvious what's going to happen.

                      Steve: You can't get that wrong really. That's part of the ES 2016 proposal and if you want to do that today you can either use Babel which will transpile async/await just directly or you can use the generators feature which is in ES 2015. Slightly different syntax but it gives you the ability to do something almost exactly the same.

                      [00:07:30]

                      Jon: Can you explain generators a bit?

                      Steve: A generator for a C# programmer is exactly like the yield keyword that came in with C# 2. So you can have a function that returns an infinite number of values. Like every time that you ask it for the next value it will give you the next thing and it will never stop doing that. That's yield in C# and generators in ES 2015 do the same thing. So you can have a function and you put a little asterisk after the keyword function to say that it's a generator and then inside there you can have yield, return, whatever. You can do that inside a while(true) loop or something like that and then the thing that calls it will receive each of these values. 

                      [00:08:08]

                      Jon: Do I just throw away promises now?

                      Steve: No, you still need to use promises. The way you can use this to make promises feel synchronous is by having some other that invokes your generator that gets back the sequence of values. Each time it gets back a value it will say "well what kind of thing is this? If it's a promise then I'm going wait for that promise to complete before I ask you for the next value." And so, if you are yielding a series of promises in effect it's like your promises are being called sequentially one after the other. Which is why your code ends up looking synchronous. It's not very easy to understand verbally but when you see it written down it's quite straight-forward and nice.

                      [00:08:53]

                      Jon: So then there was Emscripten?

                      Steve: Yeah, that's right. So just yesterday or the day before depending on timezones there was an announcement from Brendan Eich that there's going to be a new bytecode format coming to the web. So instead of Javascript just being expressed in plain text as it always has been it will be possible to describe your code in a binary format. The idea with that really is just to take all of the good work that's been done with Emscripten which compiles native code to Javascript and Asm.js which is a special subset of Javascript that can be optimized very well and say "Okay, we don't need to write out Asm.js as legal Javascript any more we can just write it out as binary code and that saves the browser a lot of work in terms of parsing." Asm.js is very fast at runtime but it still imposes a high parsing cost on the browser and that's particularly noticeable on mobiles where people complain that say a large game might take multiple seconds to start during which the browser CPU is just 100% parsing so it would be nice to eliminate that cost altogether. Also, you get smaller use of network traffic because it's very compact and also even though the semantics of web assembly are the same as asm.js initially they are open to adding additional web assembly only features in the future. So it might be possible to have new types of parallelism for example that's only possible in web assembly. If they decide that it would be inappropriate to have it in Javascript it might still be appropriate for there to be a low-level feature that other languages can compile down to.

                      Jon: Ok.

                      [00:10:39]

                      Steve: So yeah, that's all about performance really. And you can kind of do it today you know. We've got Emscripten today, you can compile pretty much arbitrary C code. You can get your asm.js - it will be very very fast in Firefox - the only browser that supports it today but other browser makers have committed to implementing that to. I know IE certainly has, I don't know about all of them. I'm pretty sure they don't really have a lot of choice these days.

                      [00:11:04]

                      Jon: Yeah.

                      Steve: That's useful for a lot of things but one thing that you can't really do very well with that today, but it's kind of promised in the future, is compile some other language, that's not C or C++, into Javascript. So theoretically you could do it because Emscripten takes LLVM bytecode and turns that into Javascript. So in theory anything that compiles to LLVM bytecode can be translated into Javascript but in practice it's much more complicated than that. So if you take a Swift application for instance, and that uses the whole LLVM toolchain, it will produce LLVM bytecode and you might be able to get Emscripten to produce Javascript but it's not going to run because you don't have the Swift runtime there in Javascript. And it's the same with Go and Rust and so-on and in certain cases people have managed to get little bits of experimental code working but really it's not a realistic option today but in the future it's quite likely with web assembly that they'll be a strong reason for the makers of these other compilers to want to support web assembly.

                      [00:12:06]

                      Jon: Ok. I believe you had tons of great demos during your talk - this was one where you said it basically very difficult to get a demo working for it. Was it that one?

                      Steve: I gave a demo of compiling SQLLite - the embedded database - down into Javascript which is absolutely trivial. It works flawlessly and you need to do nothing clever at all. But SQLite is just written with C so it's kind of the best case - a very easy thing to get working. The thing I struggled with was - yeah I did try and compile some Swift and some Go and some Rust and none of them would work. I checked all the mailing lists and nobody else can do it either.

                      [00:12:48]

                      Jon: Okay. Wow. So next there was shared array buffer.

                      Steve: Yeah. Shared array buffer. So one of the limitations that Emscripten faces today is that it can only use the features that are available to Javascript. Obviously. Because it's running on a Javascript virtual machine. And that means that there are certain types of parallelism that you can't do because Javascript has always worked hard to be single threaded. Now we've got web workers with Javascript but they are deliberately limited. With the web worker each of your workers is only allowed to access it's own private memory space and when you wan't to communicate between your application and those workers you have to post these serialized messages to say what you want them to do and they post a message back saying what they did. And that's cool but they can't really approximate the performance of native code that will spin up multiple threads that all just mutate the same memory simultaneously without any complicated message passing. So shared array buffer provides a way to pass a reference to a block of memory where you're effectively saying "I opt in to having no control over timings here. I accept the fact that anybody can mutate my memory at any time with no guarantees." And when you opt into that for your shared array buffer then you can get much higher performance in certain cases.

                      [00:14:05]

                      Jon: It's interesting we were talking to Damian Edwards yesterday about some of the server work that they're doing optimizing serving ASP.NET 5 and that was something they ran into on the server side was this kind of shared buffer and passing and minimizing the transitions and all that kind of stuff.

                      Steve: It's funny isn't it. On one side of the industry we've got the move towards functional programming and no shared state and at the other end you've got this need for exploiting all the core of many core processors with as little amount of overhead as possible where you do need to mutate shared state. So we're kind of straddling these two competing goals there and for certain applications like, for example, high performance games or something like that the shared state parallelism is clearly winning and other things like certain types of business applications functional programming seems to deliver much higher benefits than that.

                      [00:15:00]

                      Jon: Okay. I remember one of the other ones that I was actually really excited about because I'd looked at before and meant to play with and hadn't yet was Grid Style Sheets.

                      Steve: Oh yeah, yeah. So that's funny. The first thing to know about Grid Style Sheets is that it's got nothing whatsoever to do with grids. It's a company called TheGrid which, you know, is a cool name for a company but it's a little confusing when you make a product called Grid Style Sheets. Anyway the idea with Grid Style Sheets is to kind of break free from all the layout limitations of regular CSS. Any web developer in the world has struggled with making things get vertically centered or shrink-wrapping to their content. You know, that kind of thing. It's embarrassingly difficult for us, even when we consult on some of the hardest problems in the world, we can't make something centered inside something else.

                      [00:15:47]

                      Jon: And it's funny because you feel like I can describe the problem. I want this to be here and this to always be here but never bigger than the parent. Or whatever, right?

                      Steve: And that's exactly what Grid Style Sheets does. It's an example of a constraints-based layout engine and with a constraints layout engine you don't necessarily care so much about hierarchy and padding and things like that you just make a rule that says something like "My popup menu needs to be left aligned with some other element and it needs to be vertically centered in a certain area and it needs to be big enough for it's content" - that kind of thing. So it's just a few constraints you give and then the constraints solver will choose the appropriate position for it. And it's the same technique that's used in some modern iOS applications. They've got a technology called Auto Layout and that allows iOS to have things like - you'll see they've announced the Windows 8 style sidebar thing in iOS 9 I think. Well don't Apple a good original idea that one. And the way applications can fit into arbitrary sized screens is by using technologies like Auto Layout where the developer just specifies a set of constraints and the operating system positions things. And Grid Style Sheets allows you to do precisely the same thing but on the web. It's a cool, very cool technology. It's a bit experimental at this stage, I don't know of too many people really using it.

                      Jon: And it's just Javascript right? So I mean I can start using it today?

                      Steve: Absolutely. I think that the people building that would aspire to it one day maybe even being a native feature. So you have your style type equals text/gss instead of css and then you have this alternate syntax for defining layout. Which is a heck of a lot more expressive and straight forward than regular CSS in certain cases. It's also got some really cool powerful features that have got no equivalent in CSS. For example, well a simple example is that it has variables. Now of course if you're using SASS or LESS you know about variables but these are cool variables. They don't just have a specific value they have a value where you can define some rules. Like you can say this variable is positive, this variable is less than 250 but you don't say what actual value it is. The constraints optimizer will choose the value in order to satisfy all the other constraints. Also, it supports if/else conditions so you can say something like "if this element is bigger than a certain size then I want this layout otherwise some other layout which is like media queries but, you know media queries are limited to querying just the window width or some other basic thing like that. Whereas with GSS you can query absolutely arbitrary things and it's evaluated at run-time so if you mutate the DOM over time then it will update its layout accordingly.

                      [00:18:34]

                      Jon: One really nice feature I liked about it too is that it animated between. And so, I watched as you resized the window and not only would it apply the constraints but it would like really nicely smoothly move to it.

                      Steve: Yeah, so that's not actually a natively GSS feature. I just put a CSS transition rule in. So like transition * 1 second or something.

                      Jon: Smart, I like that!

                      Steve: So in an application when everything to just instantly snap to the correct position you wouldn't have that rule but it's quite sweet as you say to see things so of fluidly slide into their correct position. Which is dead easy because GSS internally is using position: absolute or something for everything so all your animations just work.

                      Jon: Now, because of things like that it might be difficult to use some GSS and some traditional CSS together.

                      Steve: Yeah, I'm sure that's probably true. Yeah. I think there's a lot of reasons why if you are adopting this you should expect to be in for a slightly bumpy ride. You will definitely be one of the first adopters in the world if you pick this up today. You'll be the one that's actually figuring out what the best practices are, what stuff does and doesn't work that well so yeah I wasn't standing up there trying to advocate that everyone should take this on today but if you've got an interesting little side project and you want to try out some really cutting-edge possible future thing then yeah it's a really fun thing to play with.

                      [00:19:53]

                      Jon: And one thing too I was all excited about it and kind of sold on it and you explained too that it can be difficult to debug. If something's not working it's like actually solving a constraint and you don't get that clear understanding of what I did wrong.

                      Steve: I could imagine some future tooling that shows you visually how things have been anchored and laid out. But at the moment it's a bit like if you give it not quite enough information it will just make some decision that might completely bizare like it chooses to position an element off the screen or something like that and it may not be obvious until you really think very carefully why it's done that.

                      [00:20:30]

                      Jon: Great, well this is fascinating. I'm looking forward to watching the video again and digging into some of this stuff. I understand you've got to run off and catch a plane.

                      Steve: Yes, that is true.

                      Jon: So thank-you very much for your time and was great talking to you.

                      Steve: Cheers.

                      21 min
                    • Herding Code 211: James Mickens on The State of Computer Security and Bitcoin and Thomas Jefferson and Internet of Terrible Things and Prawns and Oslo's Terrible Secret

                      Episode 211

                      While at NDC, Jon talks to James Mickens about his terrifying computer security keynote presentation.

                      TLDR you are doomed.

                      Download / Listen: Herding Code 211: James Mickens on The State of Computer Security and Bitcoin and Thomas Jefferson and Internet of Terrible Things and Prawns Oslo's Terrible Secret

                      Show Notes:

                      • Hello
                        • (00:24) All is lost. Listen to Danzig.
                        • Stop putting your money in Bitcoin
                          • (01:00) James says it's not a good place to put all your money unless you want to be poor.
                          • (01:42) Bitcoin is a commodity, not a currency. Investing in commodities is hard. It's not a stable place to put money. When it's going up, the smart people are taking money out.
                          • The legal system - and specifically Thomas Jefferson - won't help you
                            • (03:56) Judges and lawyers are very far removed from modern technology. Analogies to older forms of communications doesn't work.
                            • (04:44) Emojis are not telegrams. They're incredibly terrible, though.
                            • (06:30) Thomas Jefferson wouldn't help with issues relating to iPhones. If your phone broke, would you take it to a really old person for help? Because Thomas Jefferson is a really, really old person.
                            • The Internet of Terror
                              • (08:10) The internet of things is the new cool buzzword, but nobody wants to reboot their house.
                              • (09:17) The security story for the internet of things world is cartoonish. When your refrigerator starts sending spam, James will laugh at you.
                              • (11:52) The first several generations of IoT houses will be terrible. Things will get better on a relative scale, but will always be terrible on an absolute scale.
                              • Node.js
                                • (13:32) James knows he's in danger for dissing Node.js, but he can't help himself.
                                • (15:08) We'll see the rise of the JavaScript Full Stack Developer, which could be powerful, but it's also what Sauron wants.
                                • Oslo
                                  • (16:10) Oslo is a wonderful city. It never gets dark, which makes you feel safe but if James was a criminal he would be all over the extended working hours.
                                  • (16:54) Shrimps are prawns, and unlike American shrimp they have legs and heads.
                                  • (18:06) French fries cost approximately one mortgage payment.
                                  • (19:42) James speculates on whether Oslo harbors some terrible secret. The condos look like lairs for James Bond villains.
                                  • (21:11) Jon saw a building with a big sign that said Edderkoppen, which he had recently learned means The Spider. Perhaps that is Oslo's terrible secret?
                                  • Show Links:

                                    • [video + slides] Not Even Close: The State of Computer Security
                                    • [video] LIFE IS TERRIBLE: LET'S TALK ABOUT THE WEB - from Øredev Conference
                                    • James Mickens profile: All hail the Galactic Viceroy of Research Excellence
                                    • 24 min
                                    • Herding Code 211: James Mickens on The State of Computer Security and Bitcoin and Thomas Jefferson and Internet of Terrible Things and Prawns and Oslo’s Terrible Secret
                                      While at NDC, Jon talks to James Mickens about his terrifying computer security keynote presentation. TLDR you are doomed. Download / Listen: Herding Code 211: James Mickens on The State of Computer Security and Bitcoin and Thomas Jefferson and Internet of Terrible Things and Prawns Oslo’s Terrible Secret Show Notes: Hello (00:24) All is lost. […]
                                      24 min

                                    About Herding Code

                                    From the publisher's feed

                                    A technology podcast with Jon Galloway, Scott Allen, Kevin Dente, Scott Koon