We all want to build better software faster and more reliably. Many valuable simple improvements in code practice, design and process are easy to spot from outside the team or organisation but it is always challenging to know what the best thing to work on is day-to-day.
External impetuses from consultants, managers, consultant reports or even customers are not always valuable to the delivery process. Why? Because we are told that we want to achieve a pipeline of our activities. We wish to envisage new feature requests and bug fixes flowing through our pipeline effortlessly from start to finish. Flow theory tells us that we are, either individually or as a team, at peak performance when we are in ‘flow’. The pipeline model helps us reinforce this idea of flow in our conceptualised work. This makes us feel comfortable. We come up with ideas, we prioritise ideas and the ideas, bugs or features are incorporated into the software, and we deliver the perfect result.
But, of course, we know software is rarely perfect. There are distractions, interruptions and changes outside of our control. Therefore, should we model our workflow as a pipeline at all? If we accept that software is not ever going to be perfect – then why should we envisage it as a pipeline at all? Isn’t that the wrong visualisation? Haven’t we been getting it wrong all of this time?
For me, Barry M. O’Reilly’s work is closer to the mark with his philosophical stance on software residues. Residuality Theory assumes that software will always fail but some parts will remain. Those remains or residues will have already been proven to be stronger than the rest purely because they still exist. Intrinsically, Residuality Theory expects and predicts the eventual failure of a software system. If we, therefore, expect software to fail, then our whole concept of a smooth delivery of software is surely false? The delivered artifact is already imperfect because we know it’s going to fail at some point. Therefore, why should we treat elements of software delivery as smooth, perfect pipelines of work? Why should we expect it to act as a pipeline at all?
I recently saw a paper which attempted to show how a high workload on project teams could cause turbulence in the ‘flow’ of work items through to completion. While I applaud the paper and its attempt to show how high Work In Progress is a bad idea for teams – it assumes that features flow through a delivery system. I believe that features don’t flow through a delivery system. Instead features, bugs, ideas, customer feedback are all just inputs to a system which is constantly in flux. Our software is ready to fail but we just don’t know it yet.
Ok, so if we assume what I say is true – then what’s the point of building more software if it’s doomed to fail?! This sounds like a rather gloomy, nihilistic assessment of the state of software delivery.
Well, I believe that it’s just a case of not letting ourselves get trapped into thinking about software as a bunch of pipelines that work effortlessly. Let us instead consider that our pipelines are just collections of systems which are working independently to help our delivery rather than providing the mechanism for delivery itself. What does this mean?
Let’s look at the pipelines we might typically use in software delivery:
- feature pipeline (user stories, bug reports, change requests)
- build pipelines (automation for creating our built software artifacts more reliably)
- testing pipelines (test automation which proves our new builds work the way we expect them to)
- delivery pipelines (packing and delivery to production)
These all help us understand the work we have to do, and do it more reliably and more quickly. So far, so good. But if we accept that the delivered software will have faults, what pipeline are we missing from this picture? I suggest we’re missing a feedback pipeline:
- feedback pipeline (instrumentation and reporting of system and software systems to get us “ahead of the failure”)
So, what is the best advice to improve your software delivery right now? Stop thinking that software delivery is a single pipeline of work. Start thinking about how you can enhance the individual pipelines in your delivery, and also think about and prioritise a method of creating a ‘feedback pipeline’ to receive and act on system feedback before failure.
I’m sure I’m not the only person to have said this but would love to hear your views on this approach to pipelines and prioritising feedback.