Software Engineering Daily

Building a software business today requires lots of decision making. Building software isn’t just about choosing a programming language, or a framework, or a database. Developers have to choose the right cloud service provider, the best issue tracking service, the best hosted code repository, the clearest data visualization tools. We need a platform to compare the options between different products, all in one place.

In today’s episode, we talk to Ayan Barua from Siftery, a platform for software products and the companies who use them. Siftery’s mission is to help technologists find the best products for their job, whether they are a sysadmin or an advertising account manager. My discussion with Ayan covers the question of build versus buy, how to build a recommender system, and how software will be built in the future; this is also a great episode for anyone who is thinking of how to build a two-sided marketplace business.

The post Building vs. Buying Software with Ayan Barua appeared first on Software Engineering Daily.

Stack Overflow is used by developers to find out how to build software. Stack Overflow is both a tool and a community, and today’s guest Jeff Atwood has made a career out of building tools and communities. As the co-founder of Stack Exchange and Discourse.org, Jeff has been solving the problem of civilized online communication for seven years.

In today’s episode of Software Engineering Daily, Jeff Atwood talks about building online communities from the perspective of an engineer as well as a sociologist.

The post State of Programming with Jeff Atwood appeared first on Software Engineering Daily.

Internet Explorer, Google Chrome, Firefox–it’s easy to forget that these modern browsers descended from the war between Microsoft and Netscape. Today, we hear from a software engineer who was on the front lines of that war, back in 1992.

Eric Sink was one of the original developers of Spyglass, a browser that was licensed to both Microsoft AND Netscape. If you want to understand why that happened, and what the inside story of the browser wars is, and what the business of selling browser licenses was like in 1992, check out this episode of Software Engineering Daily.

The post Browser Wars with Eric Sink appeared first on Software Engineering Daily.

In this episode, Jeff and Jeremy discuss Internet.org and the criticism it has received from organizations like the EFF. Internet.org is a program created by Facebook whose purpose is to expand the reach of the internet by connecting people in developing nations like India.

However, it has come under fire for its failure to provide “the whole internet” and limiting access to a subset of websites which have to be approved by Facebook. Critics call this a violation of net neutrality and argue that it betrays what the internet stands for – openness and fair access to all destinations. This episodes jumps into the nuances of the debate, with Jeff playing counterpoint to the EFF’s stance – that Internet.org is not the internet.

Jeremy Malcolm is a technologist and lawyer who works for EFF’s international team on issues such as intellectual property, network neutrality, Internet governance, and trade.

The post Internet.org and the Fight for Net Neutrality with Jeremy Malcolm appeared first on Software Engineering Daily.

Following the successful experiment of History of Hadoop, we are doing another Saturday experiment: an editorial podcast. Let us know your thoughts via Slack, Twitter, or email!

Our podcast errs on the side of technical rigor.

Whether the topic is distributed databases, microservices, Soylent, Uber, or Dwarf Fortress, we try to separate hype from substance, deferring the narrative to the guest. With that deference there is editorial objectivity on the part of Software Engineering Daily.

One Slack channel member said “when Software Engineering Daily’s opinions are announced, that takes time away from the guest.”

Fair enough.

However, as with any journalistic organization, we have opinions. On SE Daily, we stream objectivity and batch subjectivity.

This episode is willfully subjective. It has no guest. It is a monologue editorial inspired by Developer Tea.

Immutable laws are rare in software engineering, and when an engineer claims to have found one, that engineer is usually regarded with skepticism.

General principles are more welcome.

In this post and podcast episode, I convey some loose philosophies about modern software engineering. These are strong opinions weakly held. I welcome debate and discussion.

Software is a new field and nobody knows how to do it. If someone says you are unqualified and therefore you must do maintenance work, you should question that person. We have an upside down system where the people who are paid the least do the crappiest work. They tend to be young and naive.

This is not an axiom.

The narrative that is sold to young engineers by giant companies is the following: take your $80k/yr job, do the software maintenance which makes the company $1 million, and hate your life.

After you have spent enough time in the first tier of the intellectual strip mine, we will make you an SDE 2, where you can do slightly higher level refactoring for $150k a year, which will make the giant company $5 million. This is what is called an arbitrage.

We have an assembly line mindset left over from the industrial age. Software is more of an artisanship. Don’t believe that you are a replaceable cog. Don’t believe the one-size-fits-all interview process with whiteboarding problems. These serve to grind away your individuality and make you feel like an assembly line worker.

The planning and design process is an art, but once the requirements are in place you can proceed more deterministically. The same is true for the other quantitative activities I have taken part in–poker, music, and writing. As Michael Rosenthal and I discussed, the question of art vs. science is the same question as strategy vs. tactics.

I learned this very early on playing poker when I had to leave that career, and I had tightly coupled poker to my identity. If you make your job the same thing as who you are, then your self-worth is defined by those who are judging you in your job.

Your job is a means to service your own higher purpose.

When you take an action on your smartphone, there is latency before that action is ingested.

Servers sometimes will lie to you, but servers tend towards eventual consistency. The world works the same way. In the short term, human systems lie to us all the time, but the world tends toward eventual consistency–the truth eventually presents itself.

A decent analogy is the efficient market hypothesis: slowly efficient markets are an eventually consistent process.

The world is a distributed system–what is the consequence of this? We have to do the arduous risk and reward calculations that are mandatory for every distributed systems programmer.

Long-tail failures can and do occur.

In a distributed system, we often prioritize safety over liveness. In a distributed operating system, the programmer takes all precaution to avoid data loss. Similarly, if a real life scenario presents a small probability of giant downside risk, you should take huge precaution. If someone offers you to roll a die with 1000 sides, and if you roll a 1-999, you get $100 million, but if you roll a 1000, you get your head chopped off, the upside is capped, but the downside is not.

I can estimate how significantly $100 million will improve my life, but it is impossible to quantify how bad it is to get my head chopped off. I would almost never take that offer, because the expected value is infinitely negative.

Coming back to the consequences of living in a distributed system, there is a huge range of potential outcomes in a given transaction, and assessing these long tail, massive risks takes a lot of deep thought and calculation.

This is a quote from Peter Thiel. He argues that we should think in terms of calculus rather than statistics because if you believe the world is a lottery, then you will give yourself permission to lose.

Example: a software engineer might say “I am applying to a job with 1000 applicants, so the chances I will get the job are pretty low.” That is looking at a game of skill as a game of luck, and if you don’t optimize with lots of preparation and sleep for that interview, then it’s your own fault.

If you play poker or Magic or Dominion, it is easy to blame a loss on luck, but the best players are the ones who optimize in spite of bad luck. Personally, I like playing games under circumstances when I get unlucky because it makes for more of an interesting challenge and a better learning experience.

“Shallow men believe in luck or in circumstance. Strong men believe in cause and effect.”

Luck is an idea that rescues us when we don’t do enough due diligence.

We live in a society that loves to talk about luck because it gives us an excuse when we lose. That is not to say that there isn’t luck in the world–plenty of people are unlucky. But if you are reading this, chances are you are in the luckiest .01%.

There is a long-running philosophical argument about whether or not we control our own fate. In actuality, it doesn’t really matter.

We should assume that we are in control of our own fate.

If we aren’t in control, it doesn’t change anything whether or not we assume that we are in control. But if we are in control, it would be very hazardous to our well being to assume that we are not in control.

Talk is cheap and execution is scarce.

This is why nobody cares about your ideas, they care about seeing your prototype. John Mayer said that all he had to do to become successful is finish his songs.

At Amazon, they called this “bias for action”. At Facebook, they “move fast and break things”.

That said, usually you can choose both. Long-term planning is underrated because few people take the aggressive short term actions necessary to iterate towards it. If you have never seen long-term planning bear fruits, it’s hard to see the point.

You have to figure out what to believe for yourself.

There are also lots of great people, but it is up to you to develop intuition and strong reasoning.

There is a scene from The Big Short where Mark Baum, played by Steve Carrell, is standing in front of bankers saying “the world is full of fraud, our food is fraudulent, our banking system is fraudulent, our medicine is fraudulent”, and if you look around and investigate each of these things, you do find that these systems have layers of deception and lies–software engineering is the same.

For example, if you get to a place that gives you lots of perks like free lunch or back massages, understand that they are doing that because they are underpaying you.

If you are getting $130k salary, $70k in stock options, free lunch, and free laundry, that may sound like a pretty good deal, but what does that say about how much money the company is making off of you?

According to Business Insider:

This isn’t even skewed towards how much they make off of engineers.

Software economics are unintuitive.

What is crazy about software is that it takes $0 to make new copies of a commodity that only has to be written once. This contrasts with the assembly line where each new unit takes new effort and human friction to reproduce. With software, only the first unit takes significant labor to produce.

As software engineers, we have to rethink our value as workers in a marketplace with the insane leverage of software economics.

Giant tech companies can pull off this arbitrage of underpaying engineers because engineers let themselves be seduced into a set of myths.

“The intellectual tradition is one of servility to power, and if I didn’t betray it I’d be ashamed of myself.”

-Noam Chomsky

There is a narrative of a programmer who is incapable of doing anything except programming. Some programmers talk about this with pride, saying things like “I am just an engineer, I don’t want to think about the business side of things, I don’t understand the business side of things”.

Engineers have been seduced by the industrialist’s perspective that we cannot lead ourselves, we cannot evaluate opportunity cost, and we don’t understand the market as a whole.

All of these are lies, and the world will be more efficient and utilitarian if engineers take control of their careers and start evaluating the options outside of their immediate, narrow context.

If you started in sales, you can learn to become an engineer.

You don’t need a degree–if you can do the work, you can get a job as an engineer.

In the interview with my brother Michael Rosenthal, he talked about dropping out of school and doing freelance programming, and how he has learned faster since getting out of the highly tracked education process.

In Seth Godin’s episode, he talked about the deprecated education system, and the declining worth of a degree.

Don’t let yourself be defined by your past, and the labels and messaging that society applies to you. You can take control of your life and define your future.

You should be skipping to work. We spend most of our time at work.

The job market is really good right now because our economy is being completely reshaped by software. Massive economic opportunity is being generated:

These are economic fundamentals, not spurious technical signals.

Don’t believe the ranting and raving venture capitalists arguing for lower valuations and screaming about a bubble. Don’t believe the luddite anti-technologists who sneer into their lattes.

By any logical projection, the high demand for engineers isn’t dissipating any time soon…unless the entire economy collapses in some ghastly black swan catastrophe–in which case your current job will likely vaporize anyway.

As David Heinemeier Hansson said, it is worth considering how often your job puts you in a state of flow and tranquility. If your work is stressful and entirely unpleasant, finding a new job should be a huge priority.

Thanks for reading this far. I hope this post inspires some conversation, either negative or positive and I would love to know your thoughts.

Software Engineering Daily wants to make content that is useful to engineers. Whether or not this was useful, please let us know via Slack, Twitter, or email.

The post 10 Philosophies for Engineers appeared first on Software Engineering Daily.