Getting into the FLOW: What DevOps and Microservices Need to Make Software Development Much More Productive

FLOW_cover_landscape_NO_BACK_one-lineA number of technologies, or rather techniques, are coming together to make IT vastly more productive and customer-focused than ever before. Perhaps unknown to some people on the business side of the house these techniques also make IT more adaptive than ever. But they require new ways of working.

These new ways of working are a combination of adopting the new techniques (never easy), working in new types of teams (which takes careful management), and having better metrics to manage software’s contribution to the value we are all trying to create. Often what gets lost when people talk about the software revolution going on today is another ingredient so mundane sounding that people overlook it – work breakdown. Who can get excited about work breakdown? Well, if you can’t then many of the new techniques will not work for you.

So what techniques are we talking about? DevOps, Automated Continuous Integration and Deployment, and Microservices are the ones most often talked about. These are important terms for IT professionals. They signal the ability of IT to continuously deliver new value to the business….at pace. 

Behind these terms lies a different philosophy of work, which we have called Flow in the book of the same name.


The critical advance of Flow lies in the breakdown of work into very small, discrete, adaptive tasks and the visualisation of the whole work process (including everything down to thanking people for good code!). 

We will touch on visualisation elsewhere. Here we are talking work breakdown and it requires ‘real’ Business Analysts, not novelists. However, in order to understand the contribution of work breakdown we need to introduce another term that will be new to the non-IT professional: Cycle time. Cycle time refers to the length of time it takes to complete a typical software task. In Flow we are trying to get our cycle time down to one or two days – or even quicker!

What is so important about this very short cycle time? In Agile environments cycle time can be as short as 20 days but as long as 80 to 100 days, mostly due to humongous story writing by Business Analysts looking for a Pulitzer prize.

Of course, in Agile, teams will have regular sprints in order to reach milestones in the plan but very often the objective of sprints is to accelerate towards a goal that in Flow terms is just too large to be managed properly.

In order for software teams to be organised across a time span of anything from 20 – 100 days, they will be working on branches of code that might take slightly different directions and hence need merging & integrating before being handed over to the operations team. Because of the size of a project, developers will have problems in understanding the role of their code in the overall plan. The pressure will be on to deliver on time and that invariably means that important elements of testing will be overlooked. Integrating the work of different teams can be a struggle in itself, requiring a lot of rework or churn, particularly if one team is late. Then all hell breaks loose.

The result is often that the code which is handed over by the development team is in a state that often causes operational problems. In all of this IT wraps itself in its own concerns and procedures and loses sight of the key goal – creating value for customers.

In most IT organisations, integration and handover are highly problematic at least some of the time. That has led to developer teams and operational teams having a fractious relationship. Dev wants to do new things; Ops needs stability and yet has to live with changes that may have been poorly delivered.

With Flow, it would appear that there is no branching. However, the integrations happen so quickly that in fact the branching is irrelevant and the team experiences no code collisions. If an issue appears, it’s much easier to back out the work of a few hours rather than a few weeks or even months.

Also, Businesses are now beginning to understand the benefits of close collaboration and are trying to marry development and operations into DevOps teams, multidisciplinary experts that eliminate handover risk. While this is essential in its own right, there are other techniques (such as microservices and continuous deployment) which are making it much more likely to succeed. But we still contend that you need Flow with DevOps but not traditional Agile.

The analogy we make in the book Flow is with the movie industry. Before Woody Allen sets out to make a film, he envisages every single scene and camera angle first. This great creative act of filmmaking is in fact broken down into its smallest components before shooting begins. Why? Because every single professional on a movie set then understands what she has to do in relation to every other professional – makeup, photography, continuity, sound, lighting etc.

What we now have in business is a similar creative opportunity to work better together and produce great results, day after day. But we need to take this final step of imagining all the ways we can create value before work begins. It is planning but not like we have known it.

Microservices means that we have swapped monolithic software architectures for small software packages that communicate with each other. We are not trying to layout a grand plan that has been architected by someone else. We are taking an objective, often a small objective, and breaking it down into all the discrete tasks that add value.

As the package becomes smaller we have more tools for managing granularity. One of these is the free-flowing Wall visualisations of Flow. In In Flow we break work down into those Woody Allen sized camera angles. Small pieces of work that have clear value for customers. We avoid getting caught up in doing work for purely technical reasons. The developer can see the contribution of his or her code and has a day or two to deliver. In these smaller packages it is also easier to automate the testing of code. Every work task has an automated test element integrated into it. The consequence of all this is less need for rework.

A critical factor is to get people working together, is starting with small steps, seconding a developer into the ops team; doing the standups together, being respectful of each other’s point of view, gradually merging teams when communications are looking strong and trust is developing – this is the true cultural magic behind DevOps.

The concept of Flow can also be compared with a pipeline or assembly line, continuously pushing new code into the product base. The operational side of house will be working in teams with developers so that they know exactly what is headed towards them. They can see, like everybody else, every component of the developer organisation’s work.

This is the concept behind continuous deployment to live or indeed continuous integration at every point in the development life cycle. It is facilitated by the trend towards microservices and microapps. It lends itself to working in cross disciplinary teams and it is better for customers. It overcomes many of the problems that still exist within Agile settings. To get it right involves visualising work pretty much as a great film maker would visualise his or her movie. That means more time is spent on imagining value-added tasks, even though there is less of a plan.

Detailed work breakdown gives teams a strange kind of liberty. It is creative work, you know what you’re doing and why. You know what your contribution is. Better still, the CTO and her team have a valuable metric. By monitoring a cycle time of a day or so instead of 80 days, you know precisely what’s going right and what could be going wrong.

And whilst Agile is a great step forward for most businesses struggling with Waterfall methodologies, Flow demands that everything moves at pace. But Flow is not a methodology. It craves continual improvement, a focus on efficiency and is constantly evolving in order to deliver customer value.

If a method is flawed, then every project using it suffers (and cycle times measured in weeks is an example of this). Whereas in Flow, during every project, the team challenges the Flow, makes adjustments, even maybe adds new steps or modifies existing steps in order to speed up the Flow. Hence every project can now benefit from the improvements rather than being forced to follow a method or process which is out of date by the time everyone is trained in it.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s