Software Engineering Daily

Originally published November 10, 2015

Wolfram Research makes computing software powered by the Wolfram language, a knowledge-based programming language that draws from symbolic and functional programming paradigms.

Stephen Wolfram is the Founder and CEO of Wolfram Research, and also the author of A New Kind of Science.

The post Knowledge-Based Programming with Stephen Wolfram Holiday Repeat appeared first on Software Engineering Daily.

Originally published November 17, 2015

Technical debt, referring to the compounding cost of changes to software architecture, can be especially challenging in machine learning systems.

D. Sculley is a software engineer at Google, focusing on machine learning, data mining, and information retrieval. He recently co-authored the paper Machine Learning: The High Interest Credit Card of Technical Debt.

The post Machine Learning and Technical Debt with D. Sculley Holiday Repeat appeared first on Software Engineering Daily.

There is a quote from Jeff Bezos: “70% of the work of building a business today is undifferentiated heavy lifting. Only 30% is creative work. Things will be more exciting when those numbers are inverted.”

That quote is from 2006, before Amazon Web Services had built most of their managed services. In 2006, you had no choice but to manage your own database, data warehouse, and search cluster. If your server crashed in the middle of the night, you had to wake up and fix it. And you had to deal with these engineering problems in addition to building your business.

Technology today evolves much faster than in 2006. That is partly because managed cloud services make operating a software company so much smoother. You can build faster, iterate faster, and there are fewer outages.

If you are an insurance company or a t-shirt manufacturing company or an online education platform, software engineering is undifferentiated heavy lifting. Your customers are not paying you for your expertise in databases or your ability to configure load balancers. As a business, you should be focused on what the customers are paying you for, and spending the minimal amount of time on rebuilding software that is available as a commodity cloud service.

Rich Archbold is the director of engineering at Intercom, a rapidly growing software company that allows for communication between customers and businesses. At Intercom, the engineering teams have adopted a philosophy called Run Less Software.

Running less software means reducing choices among engineering teams, and standardizing on technologies wherever possible.

When Intercom was in its early days, the systems were more heterogeneous. Different teams could choose whatever relational database they wanted–MySQL or Postgres. They could choose whatever key/value store they were most comfortable with.

The downside of all this choice was that engineers who moved from one team to another team might not know how to use the tools at the new team they were moving to. After switching teams, you would have to figure out how to onboard with those new tools, and that onboarding process was time that was not spent on effort that impacted the business.

By reducing the number of different choices that engineering teams have, and opting for managed services wherever possible, Intercom ships code at an extremely fast pace with very few outages. In our conversation, Rich contrasts his experience at Intercom with his experiences working at Amazon Web Services and Facebook.

Amazon and Facebook were built in a time where there was not a wealth of managed services to choose from, and this discussion was a reminder of how much software engineering has changed because of cloud computing.

To learn more about Intercom, you can check out the Inside Intercom podcast.

The post Run Less Software with Rich Archbold appeared first on Software Engineering Daily.

Automation is changing the labor market.

To automate a task, someone needs to put in the work to describe the task correctly to a computer. For some tasks, the reward for automating a task is tremendous–for example, putting together mobile phones. In China, companies like FOXCONN are investing time and money into programming the instructions for how to assemble your phone. Robots execute those instructions.

FOXCONN spends millions of dollars deploying these robots, but it is a worthwhile expense. Once FOXCONN pays off the capital investment in those robots, they have a tireless workforce that can build phones all day long. Humans require training, rest, and psychological considerations. And with robots, the error rate is lower. Your smart phone runs your life, and you do not want the liability of human imperfection involved in constructing that phone.

As we race towards an automated future, the manual tasks that get automated first depend on their economic value. The manual labor costs of smartphone construction is a massive expense for corporations. This is also true for truck driving, food service, and package delivery. The savings that will be reaped from automating these tasks are tremendous–regardless of how we automate them.

There two ways of building automated systems: rule-based systems and machine learning.

With rule-based systems, we can describe to the computer exactly what we want it to do–like following a recipe. With machine learning, we can train the computer by giving it examples and let the computer derive its own understanding of how to automate a task.

Both approaches to automation have difficulties. A rule-based approach requires us to enumerate every single detail to the machine. This might work well in a highly controlled environment like a manufacturing facility. But rule-based systems don’t work well in the real world, where there are so many unexpected events, like snowstorms.

As we reported in a previous episode about how to build self-driving cars, engineers still don’t quite know what the right mix of rule-based systems and machine learning techniques are for autonomous vehicles. But we will continue to pour money into solving this problem, because the investment is worth figuring out how to train the machine.

The routine tasks in our world will be automated given enough time. How soon something will be automated depends on how expensive that task is when it is performed by a human, and how hard it is to design an artificial narrow intelligence to perform the task instead of a human.

Manual software testing is another type of work that is being automated today.

If I am building a mobile app to play podcast episodes, and I make a change to the user interface, I want to have manual quality assurance (QA) testers run through tests that I describe to them, to make sure my change did not break anything. QA tests describe high level application functionality. Can the user register and log in? Can the user press the play button and listen to a podcast episode on my app?

Unit tests are not good enough, because unit tests only verify the logic and the application state from the point of view of the computer itself. Manual QA tests ensure that the quality of the user experience was not impacted.

With so many different device types, operating systems, and browsers, I need my QA test to be executed in all of the different target QA environments. This requires lots of manual testers. If I want manual testing for every deployment I push, that manual testing can get expensive.

RainforestQA is a platform for QA testing that turns manual testing into automated testing. The manual test procedures are recorded, processed by computer vision, and turned into automated tests. RainforestQA hires human workers from Amazon Mechanical Turk to execute the well-defined manual tests, and the recorded manual procedure is used to train the machines that can execute the same task in the future.

Russell Smith is the CTO and co-founder of RainforestQA, and he joins the show to explain how RainforestQA works: the engineering infrastructure, the process of recruiting workers from mechanical turk, and the machine learning system for taking manual tasks and automating them.

Show Notes: Andrej Karpathy Turk Story

The post Training the Machines with Russell Smith appeared first on Software Engineering Daily.

A popular software application serves billions of user requests. These requests could be for many different things. These requests need to be routed to the correct destination, load balanced across different instances of a service, and queued for processing. Processing a request might require generating a detailed response to the user, or making a write to a database, or the creation of a new file on a file system.

As a software product grows in popularity, it will need to scale these different parts of infrastructure at different rates. You many not need to grow your database cluster at the same pace that you grow the number of load balancers at the front of your infrastructure. Your users might start making 70% of their requests to one specific part of your application, and you might need to scale up the services that power that portion of the infrastructure.

Today’s episode is a case study of a high-volume application: a monitoring platform called Raygun.

Raygun’s software runs on client applications and delivers monitoring data and crash reports back to Raygun’s servers. If I have a podcast player application on my iPhone that runs the Raygun software, and that application crashes, Raygun takes a snapshot of the system state and reports that information along with the exception, so that the developer of that podcast player application can see the full picture of what was going on in the user’s device, along with the exception that triggered the application crash.

Throughout the day, applications all around the world are crashing and sending requests to Rayguns servers. Even when crashes are not occurring, Raygun is receiving monitoring and health data from those applications. Raygun’s infrastructure routes those different types of requests to different services, queues them up, and writes the data to multiple storage layers–ElasticSearch, a relational SQL database, and a custom file server built on top of S3.

John-Daniel Trask is the CEO of Raygun and he joins the show to describe the end-to-end architecture of Raygun’s request processing and storage system. We also explore specific refactoring changes that were made to save costs at the worker layer of the architecture. This is useful memory management strategy for anyone working in a garbage collected language. If you would like to see diagrams that explain the architecture and other technical decisions, the show notes have a video that explains what we talk about in this show. Full disclosure: Raygun is a sponsor of Software Engineering Daily.

The post High Volume Event Processing with John-Daniel Trask appeared first on Software Engineering Daily.

As the gig economy grows, that growth necessitates innovations in the online infrastructure powering these new labor markets.

In our previous episodes about Uber, we explored the systems that balance server load and gather geospacial data. In our coverage of Lyft, we studied Envoy, the service proxy that standardizes communications and load balancing among services. In shows about Airbnb, we talked about the data engineering pipeline that powers economic calculations, user studies, and everything else that requires a MapReduce.

In today’s episode, we explore the business and engineering behind another online labor platform: Fiverr.

Fiverr is a marketplace for digital services. On Fiverr, I have purchased podcast editing, logo creation, music lyrics, videos, and sales leads. I have found people who will work for cheap, and quickly finish a job to my exact specification. I have discovered visual artists who worked with me to craft a music video for a song I wrote.

Workers on Fiverr post “gigs”–jobs that they can perform. Most of the workers on Fiverr specialize in knowledge work, like proofreading or gathering sales leads. The workers are all over the world. I have worked with people from Germany, the Philippines, and Africa through Fiverr.

Fiverr has become the leader in digital freelancing. The staggering growth of Fiverr’s marketplace has put the company in a position similar to an early Amazon. There is room for strategic expansion, but there is also an urgency to improve the infrastructure and secure the market lead.

Gil Sheinfeld is the CTO at Fiverr, and he joins the show to explain how the teams at Fiverr are organized to fulfill the two goals of strategic, creative growth and continuous improvement to the platform.

One engineering topic we discussed at length was event sourcing. Event sourcing is a pattern for modeling each change to your application as an event. Each event is placed on a pub/sub messaging queue, and made available to the different systems within your company. Event sourcing creates a centralized place to listen to all of the changes that are occurring within your company.

For example, you might be working on a service that allows a customer to make a payment to a worker. The payment becomes an event. Several different systems might want to listen for that event. Fiverr needs to call out to a credit card processing system. Fiverr also needs to send an email to the worker, to let them know they have been paid. Fiverr ALSO needs to update internal accounting records.

Event sourcing is useful because the creator of the event is decoupled from all of the downstream consumers. As the platform engineering team works to build out event sourcing, communications between different service owners will become more efficient.

The post Fiverr Engineering with Gil Sheinfeld appeared first on Software Engineering Daily.

Imagine that you are a lawyer.

Your work involves managing files with dense, technical text. Your co-workers collaborate with you to accomplish a complex goal that can be broken down into smaller pieces. Your work has formal specifications, but there are degrees of freedom in how you express an idea. In all of these ways, the job of a lawyer is similar to the job of a software engineer–so why don’t lawyers use tools to improve their workflow?

As a software engineer, you have project management tools like Asana that improve collaboration. You have APIs like Stripe that reduce the time spent on a complicated implementation. You have tools like linters and source control that prevent you from making fatal errors. All of these tools save you time.

At many law firms, lawyers do not have incentive to save time. They are paid based on billable hours, not individual milestones. Historically, this hourly billing made sense–lawyers have been around since long before computers. The amount of work that might go into a legal task was hard to predict before you had computers to log data, sort documents, and standardize communications.

In contrast, a software engineer has always had the ability to automate work. That’s why (in most cases) we are not rewarded based on our time spent solving a task. We are paid based on hitting our KPIs and our milestones. With the legacy of hourly billing, lawyers can look at repetitive, administrative tasks as opportunities to make more money.

Justin Kan has been building startups for a decade, and in that time he has interacted with lots of lawyers. From incorporation to fundraising to selling his company Twitch, the interactions with lawyers consistently seemed less transparent and less efficient than would be optimal.

For an engineer like Justin, the natural inclination here was to build software and sell it to lawyers. But there would be so much resistance–you would have to convince the lawyers to change their pricing model to fixed-pricing, which would give them the incentive to buy software and work more efficiently.

Instead, Justin teamed up with a few entrepreneurial lawyers who were willing to start a new law firm from scratch, and use software on day 1. The software company is called Atrium Legal Technology Services (or Atrium LTS for short), and the law firm that uses the software is Atrium LLP. Both of these companies are very new, and were publicly announced a few months ago.

The two companies work side-by-side in undecorated office in downtown San Francisco. When I took the elevator up to see the company, the elevator doors opened and revealed two paper signs pointing to opposite ends of the office. On the Atrium LTS side of the office, engineers were writing software to extract the meaning from documents.

Today, lawyers at old law firms are paid hundreds of dollars an hour to fill in document templates by editing a text document. As the Atrium LTS software gets better, document preparation will be done through web applications, with the variable names disambiguated from the parts of the document that never change from client to client.

On the other side of the office sat Atrium LLP. The legal team was dressed a little more formally than their engineer counterparts, but there was nothing close to the formality of a traditional Silicon Valley law firm. Far from the decor of a Menlo Park law firm, the office space was actually more spartan than most well-funded startups, signaling to the employees that this is an unproven business strategy, and there is a ton of work to be done to validate it.

This sentiment was echoed in my conversation with Justin. It’s possible (even plausible) that Atrium LLP could become the biggest law firm in the world, but the road to getting there will take patience and steady execution. I enjoyed hearing Justin explain the motivation for starting Atrium LTS, and look forward to covering the company in the future.

We have done several other shows about the intersection of software engineering and law, including our show dissecting software antitrust with law professor Harry First.  To find our old episodes, you can download the free Software Engineering Daily app for iOS and for Android. In other podcast players, you can only access the most recent 100 episodes.

With these apps, we are building a new way to consume content about software engineering. They are open-sourced at github.com/softwareengineeringdaily. If you are looking for an open source project to get involved with, we would love to get your help.

Shout out to today’s featured contributor Craig Holliday. Craig has worked on the Software Engineering Daily iOS app to iron out performance issues and implement features like 2x playback. Big thanks to Craig.

The post Legal Technology with Justin Kan appeared first on Software Engineering Daily.

An engineer who wants to start a business using investment capital needs to understand the expectations of investors. The market for the business needs to be huge. The team needs to have a differentiated understanding of the market, or a differentiated product. The CEO needs to have the determination to continue operating the company even when it gets very difficult. And the price needs to be right for the investor.

Even if you are just working at a startup, or considering joining a startup, you must understand how the investment market works. From a raw financial standpoint, it only makes sense to spend your time at a startup that has equity with a high expected value. Your equity will only have high expected value if the company continues to exist long enough to have an exit–the company must either go public or get acquired.

In order to make it down the long and winding road to an exit, a technology company often needs to raise money on multiple occasions. That money is used to pay employees like you! If the company can’t earn enough revenues or raise money, you are going to get fired. Then, you may not have the spare cash to execute your stock options, and you might lose the rights to the equity that you worked so hard for.

The best way to avoid this is to learn to think like an investor–because as an engineer working for equity, you are an investor.

Semil Shah is an early stage seed investor with Haystack, a fund that he started. He also works with GGV Capital, a venture firm investing out of the United States and China. Semil has been blogging about technology for many years, and eventually evolved from a commentator to an investor. In this episode, we explore the dynamics between investors and founders of early-stage technology companies. We also explore the strange market of podcasting. Semil worked at a company called Concept.io, which was acquired by Apple for $30M.

We have done some great shows with other engineering investors like Chris Dixon and Adrian Colyer. To find these old episodes, you can download the Software Engineering Daily app for iOS and for Android. In other podcast players, you can only access the most recent 100 episodes. With these apps, we are building a new way to consume content about software engineering. They are open-sourced at github.com/softwareengineeringdaily. If you are looking for an open source project to get involved with, we would love to get your help.

The post Early Investments with Semil Shah appeared first on Software Engineering Daily.

The Netflix API is accessed by developers who build for over 1000 device types: TVs, smartphontes, VR headsets, laptops. If it has a screen, it can probably run Netflix. On each of these different devices, the Netflix experience is different. Different screen sizes mean there is variable space to display the content.

When you open up Netflix, you want to efficiently browse through movies. The frontend engineers who are building different experiences for different device types need to make different requests to the backend to fetch the right amount of data. This was the engineering problem that Vasanth Asokan and his team at Netflix was tasked with solving: how do you enable lots of different frontend engineers to get whatever they need from the backend?

This problem led to the development of a “serverless-like platform” within Netflix, which Vasanth wrote about in a few popular articles on Medium. This platform enables frontend developers to write and deploy backend scripts to fetch data, decoupling the responsibilities of frontend engineers and backend engineers.

The tight coupling of frontend and backend engineering was problematic to the development velocity of Netflix.

We have done many shows about Netflix engineering, covering topics like data engineering, user interface design, and performance monitoring. To find these old episodes, you can download the Software Engineering Daily app for iOS and for Android. With these apps, we are building a new way to consume content about software engineering. They are open-sourced at github.com/softwareengineeringdaily. If you are looking for an open source project to get involved with, we would love to get your help.

The post Netflix Serverless-like Platform with Vasanth Asokan appeared first on Software Engineering Daily.

In 2003, Paul Martino co-founded Tribe.net, one of the earliest social networking sites.  Tribe had significant traction, with hundreds of thousands of users.

In the early 2000s, hundreds of thousands of users was enough traffic to pose a company with engineering challenges. Paul had studied computer science, and was able to use his knowledge of high-performance computing to write an efficient graph database, and solve the other technical puzzles that the company faced–but the business did not ultimately work out.

The failure of Tribe made the founders even hungrier for success–and it taught them lessons that they carried into subsequent businesses.

Paul went on to start Aggregate Knowledge, a marketing technology company that sold for $119 million. His Tribe co-founder Mark Pincus went on to start Zynga, the multi-billion dollar gaming company. Another Tribe employee co-founded Yammer, which sold to Microsoft for a billion dollars.

Since his exit from Aggregate Knowledge, Paul Martino started Bullpen Capital, which makes post-seed investments. The Bullpen Capital portfolio is appealing to me–partly because of the number of Internet gambling companies. Paul and I talked about gambling and other taboo business sectors–as well as what makes a good investment in the “post-seed” category.

I enjoyed speaking to Paul because he has a straightforward, no-nonsense way of talking about things–it’s very charismatic and uncommon.

We have done some great shows with other engineering investors like Chris Dixon and Adrian Colyer. To find these old episodes, you can download the Software Engineering Daily app for iOS and for Android. In other podcast players, you can only access the most recent 100 episodes. With these apps, we are building a new way to consume content about software engineering. They are open-sourced at github.com/softwareengineeringdaily. If you are looking for an open source project to get involved with, we would love to get your help.

Shout out to today’s featured contributor Kurian Vithayathil. He has made significant contributions to the Software Engineering Daily Android app. Thanks again Kurian for your work.

The post Parlaying Failure to Fortune with Paul Martino appeared first on Software Engineering Daily.