Legacy Code: Sunk Cost or Opportunity?

Legacy Code: Sunk Cost or Opportunity?

===

[00:00:00] Richard Bown: Hello and welcome to the software delivery club podcast. This is episode 21. My name is Richard Bown and I'm your host. In this episode, I'm going to repeat a presentation that I gave to the CTO craft conference earlier this week. I was contacted by them, I think on Tuesday and the. Presentation was given on Wednesday. I had nothing to go on really at that point. So I had to put the slides together and put the presentation together very quickly. But this is coming to good time for me, because allowed me to get to a point where I can put my thoughts together. Finally, after a few months of reading and researching and thinking about things. Particularly around. Legacy and CI/CD and DevOps. So how we approach code, how we approach the legacy, how we approach our systems.

[00:00:44] Richard Bown: So in this talk, I equate legacy and technical debt to be the same thing at different scales really. I see them both as an immediate problem. as soon as you've written something in code, you've incurred debt immediately. And as soon as you've deployed something, you've incurred even more debt. The attitude, [00:01:00] your engineering organization takes to, this is therefore very important because potentially rather than paying down debt all the time or seeing as a thing that we can do occasionally, perhaps we should start to think about how we can avoid it or pay it down continuously through our everyday processes. So the next step for me is ready to work out how that works in an engineering organization and how, what practical steps we can take. But for me, this was a really interesting talk for me to give.

[00:01:27] Richard Bown: Because it did crystallize some thoughts. Bear in mind, this is new and it's a bit rough. And the people who saw the talk on Wednesday had the same kind presentation as well. I'll go through it. And then I'd love to hear your feedback.

[00:01:40] Richard Bown: So some blah-blah-blah slides to start with.

[00:01:43] Richard Bown: A little disclaimer that I've already given already about the talk, not existing this week. I've got some pictures in here, which if you're looking on YouTube, You can see, they're not great. So you might want to stick with a podcast.

[00:01:55] Richard Bown: And now we get into the meat of it. So technical debt. So technical debt is [00:02:00] a broad definition and in fact, if you look on Wikipedia, it has 15 categories of technical debts currently from coding decisions to business decisions, to architecture decisions. And those categories are pretty broad as well.

[00:02:15] Richard Bown: Therefore assuming that legacy sorry. The technical debt. Is in everything that we do, because if you look at those buckets, And you talk about it. You think about it yourself. You get to a realization that, yeah. Everything, every decision that we make incurs technical debt.

[00:02:31] Richard Bown: And legacy really is just an extension of technical debt. We get to a point where technical debt becomes so large in whatever it is that we're creating. Or have created in the past, I should say. That we call it legacy and we say, okay. Now it's time to cut it off. We start with something new.

[00:02:45] Richard Bown: Additionally, we look at the software itself, the software that we're building, the code that we're creating, but also the systems. That we're delivering our code with and going through the little example of the next couple of slides.

[00:02:59] Richard Bown: [00:03:00] Because we build from assumptions. Because we build with a model or a mental model in our head, which is sometimes, but it's never right. Is it let's face it. We try to model that we might even draw it up on a whiteboard. We might discuss it ad infinitum with our other developer friends. However, it's always going to be built-in and assumptions around how we actually wants our software to do stuff. So when it finally gets in front of users, it becomes a lot more complicated. Obviously they have things in their heads which may be they haven't already communicated to us around how they want their software to work and or they see the code or the delivered system in a way that we didn't intend it to, or we never considered it in the first place. So suddenly when our code is deployed, it becomes live. It becomes legacy in production, essentially because this is our attempt.

[00:03:53] Richard Bown: Our attempt as development team as an engineering team. To deliver something which is of use to a user. [00:04:00] And then the user responds and says perhaps this, perhaps that perhaps something else. This is the feedback is suddenly very fast. The architecture then has to react to that. We make changes.

[00:04:11] Richard Bown: Perhaps even in the first iterations of this, we create a whole new parts of the system, new systems, new subsystems get created or thrown away or rewritten. And then also there are other perhaps internal constraints. That appear. So if you want to hand off to ops, for example, there may be some constraints that that puts upon our system, which we hadn't considered previously. Security team might get involved. Along these lines. I see. That as soon as we have a deployment, we have a legacy system, and legacy and technical debt are essentially exactly the same thing. Let's look at an example.

[00:04:48] Richard Bown: In this case I used. Amplify framework, which is from AWS, from Amazon web services. Amplify framework gives a very simple way for a [00:05:00] lone developer to be able to scaffold up an entire application stack with the front end and the backend whatever you like in between as well all from a single command line and enable you to then check in changes and make deployments around that as well. So it maintains state with CloudFormation for your infrastructure in AWS, but also it allows you to iterate quickly and produce SaaS solutions quite simply.

[00:05:28] Richard Bown: the next few slides, which you don't really miss, to be honest, if you're listening to this on the podcast are very badly drawn pictures, of a developer working in this manner with amplify in the simple development case. So in the case where you're still starting with an idea . And this is just as a single developer, but still using git committing changes after the scaffolding has been created, committing it to a github repository. Which then get pushed up to and deployed as required. And this works very nicely. Feedback is given to the developer very quickly, whether things are working or not.

[00:05:58] Richard Bown: And things only get [00:06:00] complicated. Of course, when we want to get users involved. And at that point, as I mentioned other teams will get interested. A developer working in a commercial software company will have other developers around him or her as well. So they'll start to have interactions on the dev environments. They might require acceptance environments and test is getting involved and also business users getting involved or end-users getting involved.

[00:06:24] Richard Bown: And finally, of course our production environment. And all of these will need to be managed essentially either by the developer or the testers, or eventually some operators. And even if we do have a nice simple single pipeline for this, it's also interesting to see how much complexity gets built up in the environments themselves. So we have the code. Which is used to create the front end and backend and the configuration that we see the, and the states that we want the users to have, but also we have the configuration for the [00:07:00] various pieces of infrastructure.

[00:07:02] Richard Bown: And also we have the configuration for the various tools that we have alongside that. So for our CI and our CD tools, They require configuration they required metadata. And a lot of the time we perhaps underestimate the amount of time that we have to spend in configuring these tools to be able to deliver what we want.

[00:07:19] Richard Bown: Even with this simple pipeline, it becomes complex very soon. Complexity builds on top of complexity quite quickly. At this point, even before perhaps a user has even looked at the system and provided any real feedback.

[00:07:32] Richard Bown: So that's the system and the complexity that we see appearing in it almost by magic. When we get to deploy an application. And then on top of this, We look at the team itself. So the development team itself and how it has to respond to the requirements that come in from all these various other actors that are wanting to help deploy and support the application in production.

[00:07:59] Richard Bown: And [00:08:00] this is where it's interesting to look at the discussions around cognitive load, particularly recently brought to our attention through the team topologies book. Where we want to understand what makes a team work best. How can a team deliver at optimum performance? And how do we get to a point where we're actually enjoying this process too?

[00:08:26] Richard Bown: We want those teams to be able to work in that best possible way and to be able to do that, they need to have a core responsibility around that platform or that stream, but also the ability to influence it every single day.

[00:08:39] Richard Bown: You want them to have the ability to be able to think creatively the space, to be able to think creatively about solutions that they can deliver to your customer. And all of this. Does come down to how much load is on their brains, how much they are. Dragged down by the systems and the procedures that you have in place that are already support the software that they've got out there.

[00:08:59] Richard Bown: And this [00:09:00] is the essence of cognitive load. How can we take the stress away from the team so they can really perform at their best and provide a service, which our customers will love.

[00:09:12] Richard Bown: I'm reading a book called The Unicorn Project. You may have heard of it. It's fantastic. It's written by gene Kim, one of the authors of the DevOps handbook, amongst many of the things. Pretty famous software engineer. And he quotes ward Cunningham. Another very famous old software engineer. I'm saying I can say old because I'm old myself.

[00:09:32] Richard Bown: And the quote goes like this. Technical debt is what you feel the next time you want to make a change. And that really gets to the heart of it. It's the next, it's the next thing that you want to do? How do I want to affect this product, this customer experience? How do I want to improve it? And if you have a feeling of dread in your, in the pit of your stomach,

[00:09:51] Richard Bown: When you come to open that change request. Or look at, git again for the neck for the next time, or create a new branch or however you organize yourself [00:10:00] in your software team.

[00:10:02] Richard Bown: Getting ready for that change. How does it feel? That is the visceral. Meaning of technical debt.

[00:10:10] Richard Bown: Just to summarize what I've been ranting about for the last few minutes. The way I see it is that legacy is immediate. Something goes into production means we have legacy immediately. We've made lots of little decisions along the way. And we've also had multiple external systems imposing or also multiple other parties imposing their requirements and their will upon. What we need to deliver. Therefore, it is inevitable that we have legacy. We have technical debt incurred all over our code base, but also our systems. Before we even get to the first user.

[00:10:43] Richard Bown: Additionally, all future decisions are going to be based around this initial deployment, this initial baselining of our application in production.

[00:10:52] Richard Bown: And we need to get to a point where we can. Actually understand how it is possible to optimize our [00:11:00] processes, our systems, our way of working. Uh, In order to get to a point where we can deliver better change or change more quickly. So the people that matter. So all of those stakeholders that we have internally, but most importantly to our customer,

[00:11:14] Richard Bown: But of course, the closer we get to our customer, the more layers of complexity we end up with on top of our system, which then work against. That feeling of nimbleness, that feeling of speed, the. We were trying to achieve.

[00:11:29] Richard Bown: So legacy is immediate. Legacy is also everywhere. In the enterprise, in the scale-up or in the startup and every major or minor decision we make, we've seen it even from a single developer's point of view. How it's inevitably will become. Legacy. It will become technical debt. It will become something which is causing cognitive load.

[00:11:50] Richard Bown: The moment we go to production at the moment we show it to another user or another user is reliant upon that system.

[00:11:56] Richard Bown: We also find that there are sticking [00:12:00] plasters that we can apply to our organization. Making a platform team around our CI/CD tooling or around our storage or around our databases. We could build a team around that, but then silos will form immediately around those, there'll be ways of working to be agreed internally in those teams. These teams will present an external interface to say. If you want to work with us, this is the way we like to work. So things will slow things down. Make things more legacy, more integrated and essentially less good for our customers.

[00:12:37] Richard Bown: So we've seen why legacy hurts we want to fix stuff for our customers. We're in a rush. But increasingly it becomes difficult to deploy even small changes because we have so much legacy, which we build up by even getting to production in the first place.

[00:12:55] Richard Bown: So often it's simpler for us to just ignore it. We've put our heads in the [00:13:00] sand. When we say. I don't want to even think about our legacy, but we've got so much technical debt incurred in everything that's happened . What can we do to avoid it? So we'll cut it off but in the meantime, we have to support it whilst we build something new.

[00:13:12] Richard Bown: And of course we want to put all of our hopes and dreams and ambitions into the new thing that we build. So we say. It may be microservices, although there's a lot of debate about that now, especially with the new. 2022 Dora report coming out and saying that. Cognitive load with microservices. Architecture is not for beginners, essentially.

[00:13:34] Richard Bown: So it could be microservices. It could be Kubernetes, of course, which is another infrastructure platform, which could cause us a lot of cognitive load. It could be fully decoupled rather than monolith. It could be a destination for our engineers. So we actually attract more talent because we're using all these cool technologies.

[00:13:51] Richard Bown: We will open source portions of our code or all of we will become a platform for the rest of the business to use. We will finally be able to retire that [00:14:00] legacy monolith, which we all hate so much. But. I've seen this fail. I've seen this many times before. I've seen the hope get destroyed when you get to a point with the new system where you realize this complexity, that this technical debt has already built up with a new system, to a point where you're now managing the old system, as well as a new system. And both of them are complex and you have even an even worse situation than you did before.

[00:14:31] Richard Bown: So you must have a realistic plan to get to a new technology or get to a new application or get to a point where you are actually delivering more value for your customer. With less effort for yourselves. Because that essentially. Is what's underpinning this effort. We want to get to a point where we can deliver more and more value for our customer with less and less effort for ourselves and more joy in our day-to-day work.

[00:14:55] Richard Bown: Rather than just abandoning this legacy, we should accept it and move on. So can we do [00:15:00] something strategic at this point? Or is it all just tactics from here on? We can read some great books. I mentioned. Domain-driven design TDD as well. Working effectively with legacy code Michael Feathers. All of these are great tactics. But we have to get to a point where we accept that legacy and technical debt will just build up whatever we do. If we took any of these great ways of working and ensured that we followed it perhaps to the letter every single day, then perhaps we wouldn't get to a situation where too much legacy. Too much technical debt built up in the first place. However, none of us are in that luxury position. We're not typically allowed to spend too much time refactoring or thinking about how we can do something better. If it works. That's good enough.

[00:15:43] Richard Bown: If it ain't broke, don't fix it and production that's for sure. All of these terms have come about because we know through our paranoia, through our experience. That things do break when we get to changes, even though we have a CI/CD pipeline. Do we trust it? There's legacy in [00:16:00] that what do we need to invest? Can we invest everywhere in our code base?

[00:16:05] Richard Bown: In our CI/CD in our environments in the way that we manage ourselves, perhaps in our typologies, can we change that to. Or do we have to take. Step back and we'll just go, okay. We might be able to create a pick list of the things that we want to do, but we have to be clever. We have to be specific about what we want to achieve.

[00:16:24] Richard Bown: What I'm trying to say here is if legacy is immediate, Then engineers should always be invested in supporting business value. So how can we make that feeling happen?

[00:16:33] Richard Bown: And this totally is about that dreaded word culture. However, I will say that it's all about the business. It's all about how close your businesses to your engineering. If your engineers can talk about the business in the same way that the business people, the people who work on the other side, the non-engineering can talk about the business. Then that's going a long way to getting an [00:17:00] engineering. Department and engineering idea. Which will reduce any feeding that there's technical debt in the first place, because everything that we do on a daily basis means value for customers. How can we achieve that feeling?

[00:17:14] Richard Bown: Through doing some of the tactics that we mentioned before. But maybe it's just about working together in collaborative groups, a lot more closely, a lot more regularly. And also we should never stop challenging ourselves. Over the code base over the systems that we have in place. Over the feeling that we have on a day by day basis as well. If something doesn't feel right. Then we should do something about it. How can we inculcate? How can we get to a point.

[00:17:44] Richard Bown: Where we can get that feeling in our organization. That's what I'm trying to drill down to in the future. That's something. For the next few months for me to think about and to come up with some, maybe ideas around it, however. For [00:18:00] today. Thank you for your patience. Thank you for letting me work through some of these thoughts, the.

[00:18:04] Richard Bown: The talk has already. I can feel moved on. A couple of steps from the talk I gave a couple of days ago, so I'm pleased have done it. I, this. You're used to you. If you do. Like this podcast or this YouTube, then please do and subscribe. It means a lot to me when I get some feedback. So please.

[00:18:19] Richard Bown: Tell me what you think. Tell me I'm completely wrong. Tell me some good books that I should read, because I do love to read a good book around software engineering. Or anything else you can say to share. And until next time I wish you. A very good day and I'll say goodbye and good luck.