Responding to Change over Following a plan.

The Agile versus Waterfall argument divides software developers. There are those that believe iterative development to be superior way of developing software, and those that believe up-front planning to be a superior way of developing software. A couple of weeks ago I had a short but passionate argument with one of my co-workers. They are super super smart, and we each stand on opposite sides of the up-front planning divide. My colleague’s argument is that you need to plan in order to execute fast and efficiently at scale. That fast execution at scale requires considerable up-front planning.

This puzzled me greatly. Real Options shows that up-front planning is inferior as an approach because you commit early and destroys lots of options. Lean tells us that planning is a form of Muda or waste. This is particularly interesting because if people smarter than me think this, then real options could be a flawed concept and I break the model… and learn as a result. After much discussion and thought, I think I may understand the problem. The insight comes from thinking about the problem using the Cynefin model for social complexity developed by Dave Snowden.

The Cynefin model consists of four quadrants (snigger – In joke) of “Simple”, “Complicated”, “Complex”, and “Choatic” and a fifth quadrant “Unordered” for when you do not know which of the other quadrants you are in. A “Simple” system is over constrained and has cause/effect behaviour. “Complicated” is the realm of the expert. “Chaos” means there are not enough constraints. “Complex” means there are just enough constraints to create an emergent behaviour which can be nudged and tweaked into a system with desirable behviour. “Complicated” means an expert can reduce the problem to component parts with predictable behaviour. “Complex” means there are feedback loops, agents and limits that create an emergent behaviour that cannot necessarily be predicted. Essentially, the difference between the two can be two can be distilled to “A complex system interacts with its context and evolves as a result, and a complicated system does not interact with its context or evolve.”

If you know in advance what is needed, you are in a complicated system. You can specify and plan in detail to create an optimal delivery at scale (A long lead time and short duration). Hence the space shuttle. Complicated systems struggle to react to changes in context which can cause analysis paralysis. In product development, complicated systems are managed and dominated by experts. The axioms of this paradigm are untested assumptions. This is the realm of early commitment and the slaying of valuable options. As a cautionary note, these systems have the potential to suffer a catastrophic failure if they choose to ignore the context changes.

If you are in a situation where the context is dynamic and an expert cannot predict in advance, you are in a complex system. You cannot specify and plan in detail because the context changes before you can deliver. Development in a complex system consists of several safe to fail experiments followed by fast developments to enhance beneficial developments and dampen non-beneficial developments. In product development, decisions are made by studying data and the impacts of safe to fail experiments and small iterative investments. Products and their context co-evolve. The axioms of this paradigm are hypotheses which are to be challenged and tested. This is the realm where real options are an optimal strategy (Check out Dave’s paper from 1857).

So the answer is quite simple. If your product is in a complicated context, use experts and up-front planning. If you are in a complex context, create options and learn to respond faster than your competition.

A final note. If you think you are complicated context, and you are operating in a market with competition… I’m sorry, you are wrong. You are actually in a complex context.

Despite my best efforts, I now find the Cynefin framework to be a useful tool to understand* and operate** the context within which I work. If you want to find out more about Cynefin, sign up for one of Dave’s course at www.cognitive-edge.com. Oh, and refer to the quadrants as quadrants. He love’s that.

*Phenomenology
**Ontology


I am an Agile Coach. I am an Organisational Dysfunction

For nearly two years I have been helping my client do agile rather than do agile myself. I have been a coach helping teams adopt Scrum, Theory of Constraints, Specification by Example and other stuff. Although its my first attempt at “helping others do it” rather than “doing it”, I have been approaching management as a coach for almost a decade. Thankfully I am working with other experienced coaches who have been helping me learn the ropes.

For the past few months, the agile coaching team have been preparing for the day when the client will achieve sustainable agile. Sustainable agile means they will be self sufficient rather than needing to rely on external agile coaches. Thankfully, our team leader, Tony Grout, has some experience in this space and we are building “self-service” “first-aid” material for the organisation. But it’s not enough. We need the organisation to take on the yoke of coaching.

For the past few weeks I have been working with dev and test managers to prepare the way for a “Management Skills Matrix”. In technical parlance, preparing the management fitness landscape to make it less hostile to the “skills matrix” meme. (Note, So far all the managers love the idea). The management skills matrix would be a public skills matrix posted on the wall in one of the coffee areas…

Mgmt Skills MatrixThere is one big difference between the management matrix and team matrix. At skill level 3, the manager feels confident enough to coach a team in the skill. Now our target is to get at least three managers in every location who can coach each skill. As we achieve that goal, I can then retire from coaching for each skill until the organisation is self-sufficient.

This was the point that I realised that I was an organisational dysfunction.

Some of the more experienced coaches had suggested I should work for the team doing what was right, rather than work for management. It felt right because management did not have a deep understanding of agile but I had a stronger feeling that I should be aligned with management who represented the goals of the organisation. The management skills matrix helped me realise that I should not work with the team at all. Instead I should work coaching the leadership of the organisation so that THE LEADERS COULD COACH THE TEAMS. That way, there would be no misalignment. Management would know why they were doing each agile practice. There would be no disconnect between the teams and management. By training the teams, I am perpetuating a disconnect between the teams and their management… I am perpetuating an organisational dysfunction.

But training every manager in every skill is impossible. Some managers just do not want to know about the intricate details of Agile. Thats fine, training every manager in every skill is the wrong goal. The goals is to ensure that each skill can be coached by at least 3 managers. (On a process note, I’m using 3 as an illustration. The final answer might be 1,2,3,4,5  or who knows depending on context).

I should treat coaching skills in the same way I manage staff liquidity. When a coaching request comes in I should ask the managers who wants to take it on and get them to do pair coaching with me. Currently the way I work as a coach perpetuates me as the key man dependency.

It answers that age old question? Who should go Agile first? The team or the leadership?

GIVEN that management want Agile

WHEN they hire a coach

THEN the coach should start with management.

So now I have to change the way I work so that I’m no longer a dysfunction. For those of you who know me, you know how hard that will be. ;-)


Real Options versus WIP?

Recently I’ve heard a couple of people say that if you increase WIP, you increase the number of options in the system. This is a fundamental misunderstanding of the relationship between options and WIP.

Increasing WIP reduces the number of options available from the system.

Consider the following two projects, both of which have five developers who can handle two tasks at a time.

1. In the first project, each developer takes on two tasks each. There are ten tasks in progress. They commit to all ten tasks at time zero.

High WIP2

2. In the second project, all five developers swarm on one task at a time. They commit to each task, one at a time, at times 0, 1, 2, 3, 4,…

Low WIP2

In the first project, they commit up front. In the second project, they defer commitments. In the second project they do not commit early, they wait until they have finished the previous task before committing to the next one.

Of the two projects, the one with lower WIP has more options and hence more value. Stated another way, two projects that deliver the same tasks, the one with lower WIP has more value due to the options it contains that the other does not.


Pay Down Liquidity Debt (Go hire exprienced agile developers)

I was chatting to my good friend Jamie O’Shaunessey about start ups and enterprises. I explained the 20 – 20 – 20 rule* that the american army use as a strategy for any engagement. The rules are…

1. Within twenty hours, they will have special forces on site to provide recon and intel.

2. Within twenty days, they will have the navy and marines on site to control strategic positions.

3. Within twenty weeks, they will have had the whole army on site if necessary.

The key point is what are you optimising for? Initially, you want to optimise for speed. Hang the cost, hang the risk, just get eyes on site. Then, you want to optimise for risk. Rebalance the response network so it can be ready for the next crisis. Free up special forces and replace them with marines. Then cost (and risk) become an issue and you supplement the marines with regular army so that you free up some of the marines for the next action.

This maps very closely to the optimal strategy for a successful start-up. In its early days, the startup should be optimised for speed. The startup needs to get to market as fast as possible, prove the existence of the market and secure the customers. Who cares about tomorrow when you are trying to stay alive today. As a result, key man dependencies are not a concern and it likely that a skills matrix like this will exist with several significant key man dependencies.

20-20-20 Pt1

At this stage, the startup may be acquired or receive significant investment, both of which facilitate a rapid growth phase. The teenage years of a startup so to speak. This is the riskiest time for the startup. Its customers will be looking for signs of deterioration of the product if it is acquired by a new owner. At this time, the startup should be focused on risk. They should be ensuring the stability of the product (pay down technical debt) before the product grows / expands, and they should be paying down key man dependencies (liquidity debt). It is at this point it needs to bring in the marines. It needs to hire those developers with enough experience to shave the Yak. Those developers who can quickly pay down the technical debt, establish a solid test automation/continuous integration framework, establish a mature self responsible development culture and acquire the knowledge to address the liquidity debt. The established startup developers have the most options so they should be assigned no work other than to be instantly available to answer the questions of the marines. (This frees the original team members up to work on developing new markets.) After this period, the skills matrix should look something like the following.

20-20-20 Pt2

Note that there are no key man dependencies.

However Marines ain’t cheap. Anyone looking for developers with five plus years of experience in Agile development skills (Ok, Ok, I mean eXtreme Programming, So sue me) will know that they are not cheap, and they are often contractors/consultants (especially in London where I’m based. Other parts of the world may be different but I’m guessing there is a still a premium to be paid). Once your startup has reached this state, it can look to reduce the costs per head. They can start to hire cheaper developers who do not have extensive Agile experience. They can hire graduates and bring them up to speed. The team can then use the skills matrix to decide if and when to release the expensive marines.

Sadly, the common pattern I’ve observed is that startups switch from optimisation for speed to optimisation for cost, missing out optimisation for risk. How can a company spot whether it is has missed the optimise for risk step? The have the following behaviours:

  1. They have dragon slayers/hero developers. A simple skills matrix will reveal these key man dependencies.
  2. They do not have a consistent culture focused on quality and risk, with self disciplined developers and product owners who appreciate the value of quality and risk.
  3. It will take longer and longer between product releases. They will have infrequent major releases rather than the continuous delivery of small releases (i.e. They will be waterfall rather than Agile as the management will not trust the quality of the application.). Every one will hold their breath when they do a release rather than treat it as a business as usual activity.
  4. There will be (at least) two cultures. One, a start up culture focused on speed that is in constant conflict with new joiners who feel frustrated at how slowly they come up to speed.
  5. Others that I cannot think of now as I have a cold.

So what does a company do when it discovers that it has missed out the stage of optimising for risk?

Simple. Call in the Marines. Go hire enough of those experience Agile developers to make a difference to the culture and pay down the liquidity debt and technical debt. It ain’t cheap, but then again, neither is failure.

 

 

* Like all good thought leaders I never let facts get in the way of a good story. If you want to see the real thing, check out the right stuff. I cannot find the link and it would great if someone let me know what it is.


The cold war between HR and the Business

Capacity Planning helps the organisation identify the constraints in the development system, and also identify teams that are not working on strategic goals.

Capacity Plan

From the organisations perspective, the most valuable work that the Credit and Payments teams could do is to help out the Windows and Mac Client teams, or perhaps Android or WinPhone Client. Any team or individual that moved to work on an unfamiliar area of the code would likely compare unfavourably to the teams/individuals already working on that code. A team from Credit might only be 30% effective compared to the existing Windows Client team.

Any HR performance / incentives scheme that compares or ranks individuals or teams against each other will discourage the Credit Team from working alongside the Windows Client team. They will find valid reasons for not making the move. The organisation will find it hard to encourage staff liquidity. They should have a policy that encourages people to work on the right work instead. A policy that encourages collaboration.

Lets put that in simple terms. Any HR performance / incentive scheme that compares or ranks individuals or teams against each other is in direct conflict with achieving the goals of the organisation.

Recently Microsoft famously gave up stack ranking. Sadly, other organisations have not seen the light and are happy for the HR department to wage a cold war against the organisation they claim to serve.


Scaling Agile to the Enterprise – Step 1.

Stepping back from the strategic view of scaling agile, I thought I would share a very very practical first step.

GIVEN you have hundreds… thousands of developers (programmers and testers),

AND you have more than a few dozen product owners

WHEN your enterprise intends to scale agile

THEN you are going to need a tool to coordinate activity and decisions and summarise status.

  1. Everyone in the enterprise needs to use the same tool for task management. Whether it is Jira, Rally, Version One, TFS, Lean Kit, Blah, or whatever tool, the entire enterprise needs to use the same instance of the same tool. Your Agile Scaling initiative will have emergent requirements. Its hard enough getting everyone to adopt a process change on a single tool. Trying to do that on multiple tools would be impossible.
  2. The tool will need to support the entire scaled product owner process and the development process (whether it is Scrum, Kanban, XP, or whatever). There will need to be a minimum set of agreed processes that everyone will need to abide by so that the tool can be effectively to support scaling to the enterprise.
  3. Some agalistas will protest saying that the team should pick their own tools. They can but they should ensure that the corporate tool is always up to date. Given that teams are unlikely to want to maintain two tools, they should probably only use the corporate standard tools (My personal preference is that user stories and above are managed in the tool and tasks within a sprint etc are managed on a physical board).
  4. You will need to extract the data from the tool and put it into a reporting database so that you can isolate your enterprise reporting from the tool… so that you are not dependent on the tool for reporting, Also, this means you can more easily switch if you find you have selected the wrong tool.
  5. All reporting needs to be automated. You never want to be in the situation where people are manually aggregating data in a spreadsheet to get an enterprise view. This will break completely when you go into your capacity planning meeting and people are making last minute updates even in the meeting.
  6. You need to have a dedicated team who maintain the tool. You need to create the budget for this as part of the Agile Scaling initiative, and as an on-going activity. You cannot rely on a team maintaining the tool as a side project. The tool is a strategic asset that needs to be carefully controlled. Without proper management, your initiative can derailed by people adding the wrong fields or processes in the tool. The tool maintenance team need to work very closely with your Agile Coaches.
  7. Your tool and supporting processes need to be Agile. Your support team need to be able to add fields/change process in quickly as emergent requirements arise. Any tool that needs modifications to be made by the vendor to add fields etc. IS NOT FIT FOR PURPOSE. If you need to create a manual process as a work around, you have broken your process.

Explicit WIP limits destroy Options.

Zsolt Fabok, a good friend of mine, has written a blog post responding to a talk I gave where I said “Explicit WIP Limits destroy options”. Before I respond, there are a couple of inaccuracies in the post that I want to call out.

  1. In the video I said that “Explicit WIP limits destroy options”. I did not say that “WIP limits destroy options”. This is a subtle but significant difference. It is possible that Zsolt misheard what I say in the video. Instead of “Explicit WIP limits”, I advocate the use of WIP limiting policies.
  2. Zsolt says “In this context it means that “the more liquid our staff” the more work items we can work on in parallel, which means that we have more options.” This is a misunderstanding of Staff Liquidity. Staff Liquidity is about the number of types of task that a team member can work on. It has nothing to do with working on things in parallel. This misunderstanding may be based on another misguided definition of staff liquidity that is based on the number of transactions in a system. (The number of transactions is a measure of system activity, not liquidity.)

Point 1 completely undermines the argument made in Zsolt post. Typical WIP limiting policies would be no more than one EPIC (unit of delivery of value) in progress, and no team member/pair should work on more than one task at a time (Blocked tasks should be marked as such). It is vital that everyone on the team knows the value of limiting WIP.

“Explicit WIP limited systems”, or Kanban systems as they are known, destroy options for the person managing risk in the system. The problem is that this coercive management technique (not necessarily the manager*) dictate the next task that a team member should take in certain circumstances. Given the WIP limit for the “waiting for test” queue is three and the queue currently has two items in it, when someone finishes a development task, then they know that the next task they should take is a testing task. Given that they do not like testing, when this happens, then they might do other stuff until someone else finishes, takes the testing task and then they will take the next development task.

The limits force behaviour. Without the limits, team members pick the tasks that they want to. Making this visible makes it easier to identify people who do not want to perform certain roles. This can lead to a discussion to see if they need further training or they disagree with WIP limits. This is a very important option for the person(s) managing the system, as it shows whether all the team members understand and support the approach.

To be clear, having more WIP reduces options. A smaller WIP increases options. Creating a system with lower WIP and no waste (i.e. where effective swarming takes place) is not a simple task.

Zsolt has been nominated for Kanban Community’s Brickell Quay Award. Please vote for him. :-)

*Sometimes enthusiastic team members can force an approach on other passive members who are not aware of the consequences.


Follow

Get every new post delivered to your Inbox.

Join 42 other followers