Skip to content
Home » How to Embrace Complexity and Uncertainty in Programming

How to Embrace Complexity and Uncertainty in Programming

or more accurately: “How to Train Your Brain to Handle Complexity and Uncertainty”

This is part 2 of a 3-part series following my earlier post – Avoiding Toxicity. Here we start to find our place in the team and understand how our behaviours are affected by our context and how, given the challenge of writing software, we should look inward and be mindful of our moods and prejudices.

Writing software brings the best and sometimes the worst out of our brains. Writing software is unique among human pursuits. A perfect definition of art meeting science. It is limitless creativity mixed with hard but constantly moving boundaries of software, hardware and expectation. Daily, we use complex techniques and frameworks to explain how we work or to make it more transparent and repeatable. But being a human endeavour, writing software is often imperfect and we fight against entropy as much as we do ourselves to make progress.

Writing software means being faced with a blank sheet of paper. New file. What do we fill it with? We must take small steps to achieve what we set out to but our target is always just beyond the horizon.

When writing software, it’s tempting to think that we can break down or decompose a problem into little pieces which make a complete solution. However, there is often too much choice and uncertainty and too many variables not least in defining the problem space.

Faced with an unknowable target, we can only ever focus on what is happening now and next and perhaps look to the past occasionally for signs of where we should be going.

In writing this piece, I am inspired by many excellent texts by others including but not limited to Team Topologies for giving us a humanistic frame of reference, The Psychology of Computer Programming for giving us insight into how programmers behave and thoughts that have stood the test of time, and to Edsger Dijkstra for what I believe is his unparalleled approach to writing what he observed. While many writers and reader get comfort in detailed references and numbers, sometimes we need to put down the evidence and just speak plainly. This is my attempt to do that mainly because my attention span doesn’t allow me to diligently reference articles quite yet.

A typical day

On an average day, I know that my brain might work in a certain way when confronted with a problem. But day to day, I cannot be sure of how I will react to it. I might be lacking sleep. I might have exercised and developed a new perspective. Perhaps I’ve recently argued with a loved one. Or maybe I’m preoccupied with some other element in my life. Human interactions will inevitably find their way into how I attempt to tackle a specific problem even before I’ve dealt with how I will present it to my team or other stakeholders.

Some days, I want to be dedicated to writing tests. I want to write some documentation some other days. I might find it hard to concentrate on a tricky problem. Perhaps I already know how to solve it or need to research it. In 95% of cases, I will look something up to get a better idea. It might be some technical knowledge I’m lacking. There could be a new way of looking at a problem I have not tried yet. Perhaps someone I work with can offer me something – but today, I don’t feel like talking.

Decisions and branches cause complexity in the way we think and therefore the way we find a solution. The reality is this – we come up with a great idea in the shower. Our brain is working on problems when we think we aren’t thinking about anything at all.

Responding to Complexity

How does your brain handle this kind of complexity? When we’re second guessing ourselves when trying to perform hard mental effort, do you wither at the thought of it, or do you dive right in? And does your mood stay the same as you work through a challenge?

Many of us move through different moods as we take on challenges. We respond differently to the task load on our brains according to what is happening around us. Therefore, we are never truly one “type” of person.

I initially explored this theme in some small way in my post, Avoiding Toxicity. And while a supportive team structure is vital, there are also things we can do in our daily work to enable us to become more productive.

The more I’ve thought about this, the more I’ve realised how I regularly assume a different persona as I work. I am all of these types of people and more at different times. Can you recognise yourself in any of them?

  • The Challenge Lover: something is broken and I want to fix it. I want to solve the puzzle and I love the challenge of being the first to find a solution.
  • The Steady Fish: I am simply swimming with the tide today. I am not interested in making a splash.
  • The Politician: Today is a day for convincing others that we have the right solution. Today is the day we try to get agreement that something has to change. I will need to deal with people.
  • The Anarchist: I am not interested in how you did things before. Now we need to change something. We need to do something radical.
  • The Accruer: I will learn something new, but only at a comfortable pace and when necessary. I will attend a training course but I will only do it to progress in my own estimation or in the estimation of my organisation.
  • The Journeyer: I am happy to work steadily. Today is a day for pure work with little emotion. I am focussing on quality perhaps.. but not necessarily.
  • The Romanticist: I am excited to be here and I want to do the best for this piece of software. I might even daydream about new features for our customers.
  • The Learner: I don’t understand something and am keen to improve this skill.
  • The Historian: I’ve been studying this field in its historical context and believe we should learn from what was done previously.
  • The Anthropologist: I see the traces of humans in this source code and in these systems. I want to learn more. This tells me a lot about my expectations in this role.
  • The Razor’s Edge: I am using the newest technology no matter what. This legacy bores me.

There are more. I have been all of these people and more during the course of a single software project or a single project iteration or even in the course of a single day.

Our changing perspective colours our code and colours our relationships within the team we work. People come and go, bringing different perspectives, experiences and working methods. But additionally, our human context gives flavour to our work.

Before we criticise a solution or a piece of code or a decision that has been made, we should remember that the reasons might have been solid and logical at the time. The context changes and we should not forget that unless we want to repeat past mistakes.

Therefore it is sometimes good to look back, to be an Anthropologist or a Historian. Sometimes it is helpful to look forward and be the Razor’s Edge. Sometimes, perhaps when we don’t feel at the top of our game, we can just be content to be average and just do the job. Being mindful of our state, our context, our place in the world gives us reason to try something different or to just be the reliable team member.

The point is – you don’t always need to be the one pushing the envelope of quality or technology.

The Egoless Programmer

We don’t always need to strive for a better solution or look for improvements. There are plenty of examples of burnout in software engineers being caused by pushing excessively in one direction – perhaps even as one of the archetypes we see above. Without a change of perspective, we can struggle to put our decision-making into context.

A recent study of studies in Burnout in software engineering showed that:

  • Tension at work, job overload, and job demands contribute to the risk of burnout.
  • Work exhaustion as one of the burnout dimensions causes turnover among IT developers.

Therefore it is wise to look at more thoughtful and mindful approaches to writing software. The concept of the Egoless programmer has been around for a very long time. You can read more about it in the Psychology of Computer Programming by Gerald Weinberg.

From "The Psychology of Computer Programming" by Gerry Weinberg

Jeff Atwood has already done a lot of hard work listing Weinberg’s 10 commandments of the Egoless Programmer. This article is very much worth reading, not least for the comments.

Tools as our Support Network

Here is a short story for you, yesterday I was working on a piece of code which was unfamiliar but was now my responsibility. The original author had gone at great lengths to make sure that the code was clean and clear, well formatted with good code coverage. However the code wasn’t in production at the time I got my hands on it, and my job was to get it there.

The first thing I noticed was that there were several things missing which would make debugging easier – return codes, missing debug, uncaught exceptions – that kind of thing. These would float to the surface pretty quickly in any production environment. Ok, this happens, I move on and try to correct this.

Secondly, I note that when attempting to make changes, even on my branch in CI, there were requirements for formatting of git messages (conventional commits) which broke the branch and proved an impediment to progress. This was also annoying to me.

Thirdly when making certain changes there are pre-commit hooks which are reformatting my code automatically and also not allowing me to commit, and it took a while to work out why this was why it was.

Overall I had the feeling that someone had spent more time scaffolding a process in place around their code than they had spent testing it out in a production-like environment. And now it was not working correctly I was impeded from even attempting a fix until I had worked through the loop and hoops that had been put in place.

Now, you can imagine that I became a variety of personae as I worked my way through these problems. I was the Challenge Lover, the Anthropologist, the Historian, the Learner, the Accruer – I was more of these people as I adopted different mental postures to the challenge. More than that, I had to put the learning into context and not blame the author for any of these annoyances. At the end of the day, they had put systems in place with the intention of maintaining quality, portability and legibility in the codebase. I perceived them to be perhaps superfluous when the system itself still has bugs, but then what system doesn’t have bugs? When is it a good time to add code which improves overall quality? Are we committed to going fast or going with high quality maintained and a chance at improving it?

These are questions that we have to answer as a team and not just as an individual.

Therefore we should take the time to implement tools which help us maintain a standard which helps the whole team. This is one of the cornerstones of Egoless Programming. This is also one of the tenets of John Romerocode for the team.

Coming Next

In the following article, I’ll explore techniques for recognising our moods and personae as we program.