Legacy and tech debt are a fact of life in all software. So how can we identify the major causes and try to limit them in our development process?

This time I discuss the major causes of tech debt and put forward a way to deal with them at various levels in your organisation. Writing software is an intensely human activity and we'll deal with the factors that making writing perfect software next-to-impossible. For at least us humans.

NOTES

https://richardwbown.com/team-topologies/

https://richardwbown.com/ddd-refactoring-and-legacy-code/

Join my workshop on January 17th 2023:

https://richardwbown.com/embracing-legacy-and-tech-debt

Before I read the book by Kent Beck I was just thinking - pesky tests what are they good for apart from getting in the way? What’s so good about Test Driven Development?

But writing tests isn’t what test-driven development is about. It's actually about designing your code in a way that matches your expectations. It's a powerful technique that, when understood, will transform the way you write code and design software.

NOTES

Kent Beck - Test Driven Development By Example:

https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530

Github template repository:

https://docs.github.com/en/repositories/creating-and-managing-repositories/creating-a-template-repository

QUOTES

00:54 - "Because TDD is not about driving development through testing. It's about designing and architecting your application. Through writing tests." [RB]

02:12 - " I find myself going backwards and forwards, making sure I understood every single step and what it meant in that first section." [RB]

03:26 - "And so you carry on writing some more code. And eventually the whole thing becomes a great big ball of string. Lots of code and no tests." [RB]

04:06 - "It's more of a tool for a structured approach to design in the first place. " [RB]

05:07 - "Writing a test helps show that it works. In fact, the word test is really wrong here. It's more of a proof than it is a test." [RB]

06:06 - "Writing code in TDD can initially feel slow and labored but in only a short time, you'll start to notice the benefit." [RB]

06:36 - "This is almost like the reverse Conway maneuver to Goldratt's theory of constraints." [RB]

08:02 - "Whatever we do, there has to be a low barrier to entry for testing." [RB]

08:42 - "And this is the true power of TDD, rather than agonizing over design and architecture and advance" [RB]

09:12 - "TDD is an absolute no brainer" [RB]

Why do we love to build new code when we have plenty of good software that already works? Is the platform language or framework legacy? Is it because we want to learn a new skill? Is it because we are just bored of supporting the old software? 

I aim to persuade you that before you decide to rewrite even part of your application in a new language or framework, you can profit by making your existing code and existing deployments better and your organisation stronger.

In this episode I equate Legacy and Tech Debt - they're the same thing at different scales - and I see them both as an immediate problem. As soon as you've written something you've incurred it. As soon as you've deployed it, you've incurred more. The attitude your engineering organisation takes to your codebase and your supporting systems speaks of your attitude to tech debt and legacy. If you can keep your code and your systems alive through constant attention - then you minimise the effects of the debt, and you minimise the chances that something becomes legacy.

This was originally presented just this week at the ctocraft conference and I'm refining it in public as I go. Please let me know what you think and if my assumptions and potential conclusions are valid.

NOTES

Link to slides:

https://richardwbown.com/wp-content/uploads/2022/11/Legacy-Code_-Sunk-Cost-or-Opportunity_.pdf

Link to ctocraft conference: https://ctocraft.com/

Link to my book list with the books mentioned: https://richardwbown.com/resources/

QUOTES

00:44 - "I equate legacy and technical debt to be the same thing at different scales " [RB]

02:02 - "Wikipedia it has 15 categories of technical debt" [RB]

02:27 - "Everything, every decision that we make incurs technical debt." [RB]

03:19 - "it's always going to be built-in and assumptions around how we actually wants our software to do stuff" [RB]

05:59 - " things only get complicated, of course, when we want to get users involved" [RB]

07:07 - "our CI and our CD tools require configuration they require metadata. And a lot of the time we perhaps underestimate the amount of time that we have to spend in configuring these tools to be able to deliver what we want" [RB]

07:19 - "Even with this simple pipeline, it becomes complex very soon" [RB]

08:26 - "We want those teams to be able to work in that best possible way" [RB]

08:59 - "And this is the essence of cognitive load. How can we take the stress away from the team so they can really perform at their best and provide a service, which our customers will love." [RB]

09:24 - "Technical debt is what you feel the next time you want to make a change" [RB quoting Ward Cunningham]

10:17 - "Something goes into production means we have legacy immediately." [RB]

11:30 - "Legacy is also everywhere. In the enterprise, in the scale-up or in the startup and every major or minor decision we make" [RB]

12:55 - "So often it's simpler for us to just ignore it. We put our heads in the sand" [RB]

14:31 - "you must have a realistic plan to get to a new technology or get to a new application" [RB]

16:25 - "if legacy is immediate, then engineers should always be invested in supporting business value. So how can we make that feeling happen?" [RB]

17:07 - "everything that we do on a daily basis means value for customers. How can we achieve that feeling?" [RB]

Sparked by a short Twitter exchange, this time I investigate why a lot of CD implementations (that I've come across) often appear to be half-complete or just fail to achieve CD. What are the root drivers behind achieving CD and why do so many implementations fall short? Is it just ME?

It's a hard subject and one that I've debated with quite a few people. This episode is an exploration of a few themes and perhaps can offer some insights or perhaps is still a work in progress. I certainly have a few more avenues to explore around where the application stops and the CD implementation begins - plus should you design your CD implementation with that in mind.

Really interested to hear your thoughts.

SHOW NOTES

My book list including links to 'Accelerate', 'Continuous Delivery' and Domain Driven Design

https://richardwbown.com/resources/

The original tweet:

https://twitter.com/BryanFinster/status/1587589240321626113

Original post with my quote and Bryan’s discussion of it:

https://riseandfallofdevops.com/5-minute-devops-cd-is-pointless-5c906d0fd164

Bryan discusses many things including “Engineer for Release” on the No Nonsense Podcast:

https://richardwbown.com/bryan-finster-continuous-delivery-no-nonsense-podcast/

QUOTES

02:19 - "in my experience over 25 years working in software on it, I've rarely seen an organization successfully deploy a consistent CD implementation across the whole org." [RB]

02:52 - "continuous delivery is often seen as a holy grail for a firm's digital transformation" [RB]

04:42 - "the organization types should be Westrum generative. That it should be a nurturing, encouraging high cooperation organization" [RB]

05:16 - "So my point is most orgs don't support the concept of CD because they don't change. " [RB]

06:02 - "Because as every good book on the subject says without fail, there is no one size fits all solution for any of this. Plus it takes a lot of work on the journey is really, never complete. " [RB]

07:16 - "Most organizations are not software delivery centered organizations. There's just something that they have to do" [RB]

09:36 - "Pride gets in the way when it comes to building software, but also CD pipelines." [RB]

11:10 - "CD typically is created by a few heroes who understand that complex domain." [RB]

12:34 - "Similarly for CD, perhaps that's too lofty a goal to aim for directly. If we look at this huge program of work that we take on when we start to implement CD." [RB]

14:07 - " where does. the application stop and where to CD begin. But that's a, that's a topic for another time I believe." [RB]

15:47 - "engineering for release, which I really love. This is the concept that you're building your release mechanism even before you've built the first line of code." [RB]

16:01 - "Before you even write a test or a single line of code, you write your pipeline" [RB]

What is legacy? What does it have to do with building new software? Why does it annoy us as software developers? Why does it worry us as business owners or product managers? Why do we ignore it as forward-lookers?  What’s so great about technology anyway that makes us do this?

What exactly is legacy?  I believe that legacy software is not always the evil, boring thing that we believe it to be - and that before you decide on building a new software platform or product you should go through a process to define and refine your legacy and decide if building new is right for you.

In this episode, I introduce some concepts about categorising and understanding your legacy systems.

[[OR LeTS ReTIRE tHE MoNOLIth!!]]

QUOTES

01:09 - Step one for me is categorize your legacy under understanding the landscape. [RB]

01:15 - If you have a system which is just not being used anymore, it's not legacy, that's just obsolete. [RB]

02:23 - your company could be bearing its head in the sand and building something new to avoid looking at the real legacy problem. I call that ostrich legacy [RB]

02:52 - This isn't a simple exercise. It is a strategic one to help you understand where you want to go with your business [RB]

03:32 - The amount of investment or sunk cost made in a software product often puts us off from continuing to support it [RB]

04:00 - This is the sunk cost fallacy fallacy. You had an idea that replacing what you had would somehow be a magical solution to all of your ongoing support and staffing issues. [RB]

04:27 - But let's look at some of the other lies that we tell ourselves about new products. [RB]

04:52 - We will finally be able to retire the monolith, and that is the real hum dinger. [RB]

06:21 - We have heard good things about Kubernetes. We have been told that we need to cut costs. [RB]

07:12 - There is a hidden complexity in bringing something to market. and this is the 80 20 rule [RB]

07:46 - So rather than thinking about the sunk cost fallacy, think about the cost of ignoring legacy [RB]

08:28 - It's a good idea to revisit your legacy categories and your landscape and try and work out how much it will change through the introduction of your new products or service [RB]

09:35 - Take the time to continuously refine your picture of what makes up your product landscape and don't be afraid to make changes [RB]

10:33 - Let's see how Elon Musk get on with his attempts to make Twitter a better platform. If you look at how that product has evolved, perhaps there are a few legacy products and systems there that he might want to think about. [RB]

An increasing number of businesses are now actually (at least partially) also software businesses. Energy companies, financial giants, insurers, and retailers – all of them have significant in-house developer populations numbering in the thousands or tens of thousands. These companies commission, buy and even build their own software. And these are not small or insignificant efforts. They are huge in many ways, interconnected through corporate IT as global IT infrastructure. It is hard to run and support all of this software using existing techniques.

Tools and techniques are often fighting against what it means to be a software developer and provide little real value to the business either. It's time to reevaluate. It's time to take a new approach to how tools and techniques are applied in the industry when it comes to software delivery.

SHOW NOTES

https://richardwbown.com/a-manifesto-for-better-software-delivery/

Global IT budget : https://www.zdnet.com/article/cloud-computing-security-or-more-developers-heres-where-the-tech-budget-is-being-spent-next/

QUOTES

00:44 "Building running and supporting software is hard. It's complicated for individuals and software companies alike" [RB]

00:55 "An increasing number of businesses are now actually at least partially, also software businesses" [RB]

01:33 "Everyone on the boards of these companies should be aware that building and delivering software is a difficult creative and fundamentally human process that they are undertaking." [RB]

01:48 " Existing accepted techniques of project management have evolved when dealing with more predictable, externally managed vendor-run projects. " [RB]

02:27 "The current patterns of agile lean and DevOps are definitely not new." [RB]

02:49 "We talk sprints and time boxes, like we know what they mean."

02:53 "The tools market alone is worth hundreds of billions of dollars annually" [RB]

03:32 "Crazily most of these expensive tools like JIRA, Azure DevOps, github, gitlab, et cetera duplicate or replicate each other's features." [RB]

04:05 "It's therefore insane to just buy a load of tools and think "job done"" [RB]

04:36 "let's ensure that the whole organization understands, that when it comes to building software, all bets are off" [RB]

05:09 "Get the tools out of the way. And back to being what they should be ,solutions to relieve us of the complicated manual processes and not self-justifying parts of the process themselves" [RB]

I talk to Tom Kennes, sustainable cloud consultant and ambassador for the Sustainable Digital Infrastructure Alliance (SDIA). We discuss how IT and software delivery can be more sustainable and what happens when Ronaldo uses Instagram. We also talked about the impact of programming language choice on carbon footprint, the amount of energy used when watching Netflix and how to right-size your cloud infrastructure. Lots to dive into for the techies with tools that are available to measure the impact of the software you build and deliver. The emphasis is very much on making small changes that add up to a big difference, so there are some really practical tips here for individuals and organisations.

SHOW NOTES

How to join the SDIA:

https://SDIA.io/join

More on the the SDIA: https://sdialliance.org/ 

The Cloud Carbon Footprint Calculator: https://www.cloudcarbonfootprint.org/.  

The most efficient programming languages:

https://cryptomode.com/c-is-the-most-energy-efficient-and-fastest-programming-language-study-finds/

AWS and Software Power Measurement

 https://medium.com/teads-engineering/estimating-aws-ec2-instances-power-consumption-c9745e347959 

Scaphandre: https://github.com/hubblo-org/scaphandre

RAPL: https://blog.chih.me/read-cpu-power-with-RAPL.html

Solar-powered website: https://solar.lowtechmagazine.com/

Instagram footprint for Ronaldo - actually a lot larger than Tom stated: https://www.gosports.com.my/news/high-energy-one-ronaldo-instagram-post-consumes-as-much-power-as-ten-households/ 

1 hour of Netflix - newer research suggests lower numbers than Tom stated: https://www.carbonbrief.org/factcheck-what-is-the-carbon-footprint-of-streaming-video-on-netflix/. 

- Carbon Innumeracy. People tend to underestimate the amount of CO2 from a gallon of gasoline a lot: Amir, G. Carbon innumeracy. PLOS ONE 2018, 13, e0196282. https://www.researchgate.net/publication/324932129_Carbon_innumeracy/link/5aebc7ea0f7e9b01d3e0652f/download

QUOTES

2:05: "I think sustainability is, overall the best thing you can do. Anything that's more sustainable will, in the end benefit society." [TK]

4:05 "First you think cloud is a good thing, (and) cloud is a good thing. Then you notice that there are some, oddities in there. They notice that why aren't we tackling those things?" [TK]

4:33 "he real question here is how can we incentivize companies to start reporting on their digital footprint?" [TK]

4:49 "cloud is the least welcome environment where you can measure these things" [TK]

5:11 "sustainable IT attempts to enable companies and developers to start reporting on their GHG emissions, their carbon footprint, their energy consumption" [TK]

5:25 "if you can enable developers to incorporate their footprint in their decision making, you are enabling such a large workforce to do more green, to do better in that context" [TK]

7:30 "the easiest thing you can do, which also saves you money in the end, is just right-sizing your cloud basically." [TK]

8:27 "There's quite a famous kind of league table of programming languages and showing how green they are in comparison or how much energy they use." [RB]

9:11 "python has a energy consumption as 80 times that of C#" [TK]

12:41 "[The SDIA is] one of the biggest nonprofits in the European Union that focus on sustainable IT" [TK]

13:09 "in the end it's about showing that you care for the environment, which it's something that investors are looking more and more for" [TK]

15:36 "there is this one tool which focuses on performance monitoring counters . Which basically directly come from a CPU and say how much energy a CPU is using. " [TK]

16:33 "You can deploy it on your Kubernetes cluster that exposes metrics to Prometheus that you can use from there. They also have some other tools as well that they use to even calculate embodied carbon" [TK]

18:03 "It really brings this quite abstract idea of the sustainable IT into an everyday life of engineer developer." [TK]

18:39 "if you look at the total energy consumed by IT we are somewhere at six, seven, 8% of the total energy consumption" [TK]

19:17 "if we were to change our ways, we can actually solve quite a big part of this puzzle next to, of course, supplying the tools to make other sectors as, as well more sustainable" [TK]

19:57 "there have been predictions that put an hour of Netflix equal to 30 kilometers of driving on an electric vehicle" [TK]

20:34 "Ronaldo, when he posts a picture it's often seen by about 300 million people. The overhead in terms of energy consumption that is implied there is about equal to the yearly consumption of one Amsterdam household." [TK]

21:30 "So it's incredible how little we actually know about our daily life carbon footprint, or emissions" [TK]

When was the last time you did a Minimal Viable Product that was actually minimal? How often do we get it right and truly focus on delivering something of value to the customer?

On this week's podcast, I riff on the idea of how to get a true MVP, not only past the idea stage, but into production no matter what type of company you are. By looking at the constraints that we have in play at any one time, we can determine before we get started with an MVP how it might be rolled out.

We also touch on the importance of marketing for your MVP and/or product – and ensuring that you have a customer base that wants it before you get there. How you can even use your compliance journey to road-test your idea.

Join me for this discussion and, as always, let me know your thoughts about what’s gone wrong and what's gone right with your MVPs!

SHOW NOTES

This show inspired by some systems thinking, some writing and also the theory of constraints.

https://richardwbown.com/how-the-mvp-has-lost-its-meaning/

https://en.wikipedia.org/wiki/Theory_of_constraints

https://richardwbown.com/approvals-come-first/

QUOTES

00:49: “Much is made of the MVP, the minimal viable product. But so rarely is it executed in a meaningful and valuable way.”

02:02: “Using an MVP to road-test new tech is not always a smart move.”

03:19: “architecture just means big decisions that you're probably not going to undo later.”

03:59: “There is a tension here between architecture and MVP”

06:35 - “By narrowing our proposal for our MVP. We stand a greater chance of getting it approved and rolled out in order to gather feedback”

07:28: “The reasoning behind the processes that build up around change control are all there for sensible reasons.”

08:15: “Whatever of path is open to you, your MVP won't deliver unless it has users”

08:49: “Don't leave the marketing to last or as an afterthought. And this applies not just to startups or individuals, but also to the scale-up or corporate world”

In this episode we explore the way that technology, tools and architecture can affect the product that you are building. As engineers do we need to think more about product side? As product marketers what can we do to understand how the technology choice informs or limits our product possibilities? How do we deliver software in an open source project vs how do we deliver software in a corporate banking environment? What are the constraints that we need to be aware of in our decisions and how the customer need informs that. How we need to enjoy what we’re building - and make sure that we continue to enjoy building it as an individual, a team or as a company.

Once again I really enjoyed recording this one and asking myself some hard questions. Perhaps you have more questions that from it? Please get in touch with me via the Software Delivery Club. I’d love to hear from you!

https://softwaredelivery.club

SHOW NOTES

Dave Farley - Modern Software Engineering (https://twitter.com/davefarley77):

https://www.amazon.com/Modern-Software-Engineering-Discipline-Development/dp/0137314914

April Dunford’s Obviously Awesome (https://twitter.com/aprildunford):

https://www.aprildunford.com/obviously-awesome

Rosegarden Linux Audio and Midi Sequencer and Notation Editor:

http://rosegardenmusic.com/resources/authors/

You can also download a compilation of my best blog posts in PDF form. I call it:

A Systems Approach to Software Product Delivery

Making software is fun. Running software can be less fun. How do you do both and not go insane? How do you make sure you're not working all the time to keep your (virtual) baby up in the air?

The same rules apply whether you're a startup, a scale-up, a corporate, an individual, an open source project contributor. You're part of your own support problem.

This time I discuss what it's like to be a coder who wants to have it all - the good times and the bad times - but wants to limit the bad times to working hours only. I hope you enjoy it as much as I made recording it.

NOTES

XKCD 242 - The Difference:

https://xkcd.com/242/

Brian Scanlan about Intercom's approach to on call.

https://www.intercom.com/blog/rapid-response-how-we-developed-an-on-call-team-at-intercom/

And he talked about this recently on Adventures in DevOps:

https://topenddevs.com/podcasts/adventures-in-devops/episodes/reducing-on-call-engineer-burnout-with-a-volunteer-management-infrastructure-devops-121

Inspiring music that helped me write this:

https://en.wikipedia.org/wiki/Milk_%26_Kisses

https://en.wikipedia.org/wiki/Reading,_Writing_and_Arithmetic

QUOTES

01:25 - How did it feel the first time you wrote something in a computer language that maybe you just learned and it worked perhaps first time

02:50 - We're in that space where creation is flowing. But at some point reality bites. Then we need to do something with this thing that we've created.

5:05 - What is happening tonight? Mummy, why can't we go out to the zoo this weekend? I know guilt right. The point I'm really laboring here is that one should be proud of your creations.

05:18 - All of our creations need us. All of these things are calling for our time.

06:48 - Some of the most successful systems on the planet have been built on a knife edge of functional validity.

07:43 - Your job is therefore to minimize that risk and ensure that you can roll back or fix forward with the least possible effort.

09:21 - Be aware of your legacy as a coder. Think carefully about how your persisted data will be managed.