It all started as a simple error message from a piece of software that had run seamlessly thousands of times over the past two years. And yet, once a week in late 2016, this error message would rear its ugly head. At first, it was thought to be a user error. But after some testing, it was clear that we were dealing with a software bug. Thus began a month-long hunt for this elusive bug, becoming as much a personal vendetta as a solution to an intractable problem.Software bugs are a fact of lif...

Most companies don’t discover they have been hacked until months later. But imagine catching a hacker red-handed? Your course of action would be very different. Instead of dealing with the aftermath, your focus would be on urgently identifying the source and then blocking the hack. This is exactly what stack.io CEO and Founder Hany Fahim did when dealing with his first hack, back when he was a young Systems Administrator. Yes, this episode tells a tale from some years ago. So long ...

stack.io CEO and show host Hany Fahim puts on his Sherlock Holmes hat in this episode of Tales from the Ops Side. He's investigating a missing file of sensitive data, while his client and their partner point fingers at each other over fault. This client had an agreement to provide sensitive data on a nightly basis to a partner. This was automated, the process called a cronjob. Hany's lunch was interrupted in late 2017 when an urgent request came in from this client.The problem: One night the...

Host Hany Fahim tells an intriguing tale of the global ripple effects of a political event and the phenomenal power of social media. Hany was - somewhat successfully – at home assembling a chair one evening in 2015. Suddenly, he received news of a huge spike in traffic. Normally, this would point to a number of things, such as a marketing event or even an attack—but nothing seemed to add up. Hunting for answers, Hany traced the users’ IP addresses. Interestingly, they all seemed to lead back ...

Host Hany Fahim tells a roller-coaster tale of an emergency situation that occurred one beautiful summer evening in 2012. When multiple client websites his company supported mysteriously went down without warning, Hany raced against the clock—quite literally, it turned out—to get everything back up and running.Although his alerts indicated that high load was at the heart of the issue, when Hany checked things out, he was in for a surprise. Nothing seemed to indicate anything should be wrong. ...

Host Hany Fahim tells the fascinating and sometimes frightening story of some massive DoS attacks that knocked clients offline for extended periods of time, along with ransom letters demanding thousands of dollars.Hany and his team spent hundreds of hours dealing with the attacks, which led to a Denial of Sleep, too! This is a desperate, edge-of-the-seat tale that makes it clear that only those with nerves of steel should get into the Internet Ops business! And of course, the work demands ing...

Welcome to Tales From the Ops Side, a new podcast show exploring Internet Ops.Tales From the Ops Side is produced by stack.io and narrated by stack.io’s founder, Hany Fahim. The show is an exploration of Hany’s adventures in running stack.io. Hany delves into the latest technologies, trends, and challenges in the business of building and operating web applications in the public cloud. If you work in DevOps or Internet Ops, you’ll know exactly what Ops Side means. For those who don’t, DevOps i...