Monthly Archives: December 2016

Why Building the Wrong Thing is so so easy

One of the key risks that organisations need to address, is building the wrong thing. No one willingly builds the wrong thing, so why do Product Owners build the wrong thing?

The first question to understand is “What is the wrong thing?”. The wrong thing is building anything other than the right thing?

So the second question is “What is the right thing?”. The right thing is a subjective decision based on the context of the organisation and the needs of the customer. At least Product Owners always prioritize the building of what they think their customers or organisations need. Or do they?

There is a nasty cognitive bias built into most organisations that Executives need to actively manage. That bias is your architecture and organisation structure. Everyone knows that it is easier to deliver something if only your team is involved. They know that involving another team in their department is harder so will try to avoid doing that. They know that involving another team in another department or division makes it even harder, almost impossible in fact. So as a result the way that many product owners approach the problem is not to ask “What does the customer need?”, they ask “What does the customer need, that I can do with my team or another team in my department?” As such, the Product Owner is more focused on their capability than the needs of the customer. They will (often unconsciously due to the culture) pre-filter those things they know are hard to deliver.

So the answer is simple, the reason why building the wrong thing is so so easy, is because building the right thing is hard.

Some Agilistas around the world have attempted to avoid this problem by creating “autonomous product” groups such as Spotify’s Tribal Model or SAFE’s Agile Release Train. My experience is that no matter how hard you try to architect your product solution to remove dependencies, there is always a customer or organisational need that crawls out of the woodwork that requires work by two or more groups. There is always a new regulatory regime, or a new way of servicing the client, or a new operating system that requires coordinated changes across multiple “autonomous product” groups.

So instead of trying to build autonomous product groups, or constructing projects from scratch, lets consider a middle ground. What would that look like?

  1. Create long lived teams based on a carefully (and continuously) thought out but dynamic architecture. Smaller components supported by a handful of teams are more stable than massive chunks of the business.
  2. Use Quarterly Capacity Planning to optimise the delivery of value from the portfolio, and make it easy to pull together teams needed to deliver the customers needs. Ensure that all of the business have an agreed ordered backlog.
  3. Understand that pulling together a group of teams who do not normally work together is constructing a value stream from scratch. Get the teams to collaborate and coordinate using “scrum of scrum” stand ups and regular retrospectives (rather than plan up front). Assign responsible to facilitate the collaboration and coordination to an individual for each piece of work. Make it easy to reassemble groups of teams by doing it frequently as required by the work rather than try to avoid it.
  4. Build what the customer and organisation need rather than build what is easy.

This was the approach Skype took to managing the risk of building the wrong thing. We may not have built the right thing every time, but at least we did not build the wrong thing because it was easier than building the right thing.

Its another genius thing worth studying that Mark Gillett created.

Advertisements

All Estimates are Waste, some are useful

Estimates are crack cocaine for managers and executives. Estimates are an emotional crutch that give managers an illusion of control over an inherently risky process. As well as being an expensive waste, estimates can drive unhealthy behaviour in teams. The problem with estimates is that there are different forms of estimate that are appropriate for different decisions and different levels of understanding, and unfortunately managers do not always use the appropriate estimates.

To begin with, an estimate is a “tea bag”, it is part of the solution needed to create an output for the manager. The output is required to satisfy a need for the manager. There are two outputs from an estimation process:

  1. How much will it cost?
  2. When will it finish?

In actual fact, you need to answer the second question before you answer the first.

The cost of an IT investment is the duration of the investment multiplied the day rate of the investment. Consider a team of eight developers who spend half of their time on support and half on new investments. In addition to the developers are a product owner, UX, managers, etc. amounting to a further sixteen head count on the project. If the investment is estimated to run for a further thirty days, then the cost will be 30 * 20 person days = 600 person days. i.e. All the people involved in the investment rather than just the four developers. That is why an understanding of Theory of Constraints is so important to managers, especially those who have been trained in cost optimisation which tends to create constraints and increase costs.

So the cost of the based on the duration of the investment, or rather, when will it finish. How do we determine that? There are two measures we can use:

  1. Count of stories
  2. Story Points

According to the work of Todd Little and Troy Magennis, there is little extra accuracy using story points over the count of stories. However there is a real advantage of the story count as it is available much earlier in the process than the story points. A reasonable estimate of the count of stories is normally available very early on in the process. Story point estimates are normally available about a sprint before the story is developed. That is, after the product owner has worked with the UX and Data dudes in the Product Owner Squad to create the prototypes, test them with users, write the gherkin format acceptance criteria and then slice the epic into stories that can be estimated by the team in the product refinement session. The alternative is to create a whole bunch of pure waste to create faux story point estimates much earlier than they should be available. Faux story point estimates that are not following the normal process and are thus not really story point estimates, and provide a false level of comfort.

Estimating when an investment will finish is best done by looking at a Cumulative Flow Diagram and projecting based on the rate stories are being created and completed. Consider the following Cumulative Flow Diagram. The black dotted line is “now” and the red dotted line is when the investment is due to finish. The red dotted line is calculated by projecting the rate at which stories will be created and finished. (Troy Magennis’s “Cool and Fabulous Spreadsheet” does something similar with numbers instead lines, and in a much more grown up way using statistical confidence intervals).

screen-shot-2016-12-28-at-11-49-27

A week later, the manager looks at the CFD and sees that the estimated finish date has moved because the rate at which the stories have completed has increased. This may be because the amount of support work has reduced due to higher code quality, or because the team are improving and going faster. The manager can investigate and potentially identify changes to the process to improve the estimate of the number of stories.

screen-shot-2016-12-28-at-11-49-41

A week later (in an alternate reality), the manager looks at the CFD and sees that the estimated finish date has moved because the number of stories has increased. This may be due to unforeseen functionality, or because a story was more complicated than expected and needed to be broken down. The manager can investigate and potentially identify changes to the process to improve the rate of delivery of stories.

screen-shot-2016-12-28-at-11-51-12

A week later (in yet alternate reality), the manager looks at the CFD and sees that the estimated finish date has not moved because the number of stories has increased but that has been offset by an increase in the rate that stories are completed.

screen-shot-2016-12-28-at-12-05-20

As well as understanding that there is a change in the estimated finish date, the manager can understand the source of the uncertainty that has created the change.

Now lets consider the same graphs drawn using story points. To begin with, we need to get some “experts” in room to estimate all of the stories, because we would not want to get the whole team to give an estimate as that costs too much. That means we have guess what the stories will be (before user testing and all that). Our “experts” will only be too pleased to provide a story point estimate, even though it is not a story point estimate. It is an expert “estimate” rather than one by the team doing the work, and it is not based on a “Definition of Ready” story, its a faux story point estimate. This means at some point during product refinement, the estimate will change to a real story point estimate and we will have some stories estimated using story points and some using faux story points. This mixture of story point methods is misleading and as a result, the manager will pressure the product owner to complete all the stories to a state of “definition of ready” so that they can get a reasonable estimate. There is no sensible* intervention that a manager can do to improve the quality of the estimates.

( * We could adopt a waterfall approach to creating stories. In order to improve the estimates, we could increase the risk and cost to the organisation substantially which isn’t really sensible ).

Even worse, there are now three reasons for a change in the estimated finish date, as well as the number of stories created and completed, we now need a third dimension for the change in story point estimates. In other words, to understand what has happened, the manager needs to study two graphs** and the difference between them. What was obvious on the CFD, has now become complicated to understand and needs an “expert” consultant to explain. This lack of transparency leads the manager to fail the Agile Risk Framework Audit that requires them to know and understand what is going on in their investment.

( ** I tried to think how you would represent in a useful way this but gave up as it was too complicated, and it was silly anyway)

One day, the manager has a revelation! Instead of using faux story point stuff to estimate the finish date (and hence cost), lets encourage the product owners to write stories of a broadly similar size. From now on, the largest story allowed will be an “eight” (or a “five” for teams with one week sprints). No more stupidly massive “thirteen” or “twenty” stories. That will mean I can get my estimates for free without any stupid and wasteful “faux story point” estimation sessions.

So if you consider yourself a competent manager, go read Todd Little and Troy Magennis stuff on estimation and prediction.

Todd Little Key Papers

 

 


Simple OVER Complicated

Mark Twain famously wrote “I didn’t have time to write a short letter, so I wrote a long one instead.

A more contemporary version might be “I didn’t have time to write a simple depiction of Agile, so I wrote a complicated one instead.

A more truthful contemporary version be “I wanted you to think of me an expert so I created a complicated depiction of Agile, especially as I did not have enough experience to create a simple one to handle the complexity of the context.”

This might explain the SAFE diagram and thw Deloitte Agile diagram.

Simple in Practice

Simple is vital for adopting Agile at the organisational level. Tony Grout, Lisa Long, myself and many others spent significant effort at Skype implementing and supporting a process to coordinate two hundred product owners and three hundred development teams.

Initially people struggled with just understanding the process. Then once they understood it, there was a desire to improve the process. This usually involved adding to the process, making it more complicated.

Tony, Lisa and I had a single design principle to guide us. The process should be so simple that no one should be able to use the process as an excuse for not following it. Anyone not following the process would look foolish if they failed to do it properly. This was because we knew there might be people who opposed the introduction of the process.

Opposing complicated-ness was difficult and we had to be constantly vigilant. For eighteen months, the process evolved. Each retrospective, the group would say the best thing was the “preparation” and the thing they most wanted to improve was the “preparation”. The initial meetings took three days with approx sixty people face to face in Tallinn, London and Redmond. The last meeting I’m aware of involved about twenty people on a conference call for two hours. Much of the quarterly meeting was moved to the “preparation” activity. We had started trying to create a week by week plan for the quarter with everyone in the room. We dropped the planning and focused on creating an ordered backlog based on the available capacity in the teams.

Although the tight knit team running the process increased the level of automation, the product owner team had only two simple instructions:

  1. To get an estimate from the product owners of all parts of the organisation needed to help them deliver their initiative. This meant as a minimum they had to have a conversation with the other product owner. We just did not specify what the conversation covered. We gave guidance that the conversation should last about ten minutes, and the product owner providing the estimate might want to discuss it with the tech lead and test lead, but not the team. We faced a constant challenge with teams wanting to provide ever more accurate estimates.
  2. They had to create an ordered backlog based on the capacity available to the teams. We did not tell them how to come up with the ordered backlog, only that they had to come up with one.

The success of the process was based on a simple process that two hundred product owners followed.

The key point is that there is a natural tendency towards complicated and it requires effort to keep things simple.

Simple in Communication

Simple communication requires practice and preparation. Communication goes through three stages, excitement, boredom and effectiveness.

  1. Excitement – During the excitement phase, we love a new idea and every time we explain it, we embellish the explanation with extra details and anecdotes. Our description of the idea becomes more and more complicated and we feel the need to add more and more “important” details.
  2. Boredom – After we have explained the idea a number of times, we get bored of it and start to drop the “important” embellishments. Our boredom with the idea leads us to drop truly important details to the explanation which results in confusion and additional effort. We learn what is genuinely important to our explanation which cannot be dropped, and what is unimportant and can be dropped. Our laziness leads us to the perfect explanation.
  3. Effectiveness – Eventually we reach the point where our laziness has led us to an explanation that is as simple and short as possible.

This process requires the need to practice an explanation many time. The best way to practice is one on one. People are less likely to ask clarifying questions in a group, and they are also more likely to ask challenging questions. You need to practice with people who do not know the material, and people who know it well… Both groups will give different but valuable feedback.

So aim for simple, and understand that complicated is a sign of a flawed understanding. As Einstein said “Everything should be as simple as possible, but not simpler”.


The Executives New Clothes

There is no reality. Everything is an abstraction. Your reality depends on which abstraction you choose to believe in. The decisions you make depends on your reality. The decisions lead to your behaviour. Changing the way you look at the world, changing your phenomenology can be a scary thing. In order for a coach to help a manager learn a new way of looking at reality, they must first build a relationship of trust with that manager. The trust is necessary so that the manager will trust the coach will be able to get them back to safety.

Lets take a look at a small example of this. One of the main ways that senior managers look at projects is the status report. The status report is the way that projects make themselves visible. Status reports are one of the big differences between traditional and Agile projects. Traditional projects present a static view of status whereas Agile projects present present a dynamic or graphical view of progress, lets consider the first derivative or “velocity”.

A traditional status report might say that 90% of stories are complete. This sounds pretty good doesn’t it? However, lets look at the dynamic view of progress using cumulative flow diagrams (CFD). Consider two different “Epics” where 90% of stories are complete.

In the first CFD, risk averse development team, the team have completed the easiest, least risky stories first. They are left with the hardest riskiest stories and it is impossible to predict when the last 10% will be completed.

screen-shot-2016-12-23-at-08-33-39

In the second CFD, risk managed development team, the team have completed the hardest riskiest stories first. They are left with theĀ easiest least risky stories and it is easy to predict when the last 10% will be completed.

screen-shot-2016-12-23-at-08-36-38

The point is that a static view of 90% complete hides the fact that the project could be in a really good place, or in a really bad place. Revealing this to a manager has no real up side for them but may demonstrate that a “successful” project is actually in a really poor state. Viewing the poor state for the first time this way, there are a number of ways that the manager and the culture they operate within may react.

  1. They accept the new reality.
  2. The reject the new reality.

From the coaches perspective, they are hoping for option one. This is only likely to happen if the manager is confident that they can survive the revelation to their management. The ability to accept a new reality is based on how safe they feel accepting the new reality. That is either because they have the options they need to survive and thrive in the new reality, or the culture they operate in is accepting of failure, and the discovery of new contexts. The new reality may be an opportunity for the manager if they are able to cope with it, and especially if their competition in the organisation is unable to.

In the event that the manager does not have a solution to the situation, they are going to have to depend on the coach to help them out… And that is only going to happen if they trust the coach. If they do not trust the coach and their ability to help them, they may reject the new reality, and the coach at the same time. They may attempt to hide the new reality from their stakeholders and fix the problem with their existing trusted colleagues.

Introducing a new reality into an organisation is risky to the individuals who experience the new reality, especially if they do not have the options to handle what they perceive. The person showing them the new reality is equally at risk if they do not have a trusted relationship with them. However the “new” reality has always been there. The executive has been wearing new their new clothes, they have been in the altogether the whole time. The difference is they are now aware of the situation and can potential react with new options.

As Dan North puts it “You go into a darkened room and turn on the lights, there are four tigers in the room. There were always four tigers in the room, its just you could not see them until now.”

Please don’t kill your coach because they switched on the lights.


The hedgehog, the fox and the CIO.

I was chatting* to Tony Grout yesterday and he introduced me to the Hedgehog and the Fox concept… The fox knows many things, but the hedgehog knows one important thing. The IT Risk Management Framework is a fox thing, it is a super clever subtly complex framework for clever foxes to provide real governance and control over IT investments, change culture, and keep clever foxes happy, all at the same time. Its just tricky for CIOs who are managing “Hedgehog” organisations who can only know one important thing.

Agile is a fox strategy which attempts to wrap up decades of good practice into a single single Hedgehog term. The only problem is when you try to execute on the Agile hedgehog strategy you discover the fox strategy is hidden inside it. Even worse, you discover that no one can agree on what Agile is. Is it SAFE or LESS or the DADDY? Oh no, Its not even a fox strategy, you just unwrapped a whole skulk** of fox strategies all fighting and nipping at each other.

Instead, lets look at the IT Risk Management Framework which consists of four drivers:

  • Deliver Value
  • Sustainable Quality
  • Reduce Lead Time
  • Manage Risk

Lets consider each one at a time.

  • Deliver Value is really a business concern. If IT investments are not delivering value, the CIO can offer advise but fundamentally they are going to be politely shown the door back to the IT department ghetto if they try and take control. The business will handle this, and the CEO will own the definition of value.
  • Sustainable Quality is a must. If the products do not have sustainable quality from the customer’s perspective, the customers will leave. Once again, the business will handle this.
  • Reduce Lead Time is under the control of the CIO. By control, I mean responsible, and they can rightly be held accountable by the CEO for this key metric. Furthermore, they tend to own all the resources to address the significant parts of the value stream.
  • Manage Risk is best done by having short lead time, and with instigating a culture of transparency. So Reduce Lead Time gives you a big chunk of Manage Risk as well.

So in summary, a CIO’s hedgehog strategy is Reduce Lead Time because it gives you most of Manage Risk, and the business leadership will cover Deliver Value and Sustainable Quality. Something that Andrew Green, Al-Noor Ramji, JP Rangaswami and Lee Nicholls have been doing for years.

Merry Christmas CIOs, Reduce (Weighted) Lead Time becomes your measurable transformation strategy.

P.S. Obviously Lead Time is not an effective metric as it cannot be scaled, and hence we use Weighted Lead Time instead.

* OK, so this is mostly Tony’s stuff but I’m gonna write it down because he most likely will not.

** Who knew, the collective nouns for foxes are Leash, Skulk, Earth, Lead, Troop!


context Risk Management

context is the most important consideration for any organisation engaged in a Organisational Agile Transformation. Most teams of seven plus or minus two are fairly similar. The issues they face are fairly common and are fairly well served by established Agile practices like Scrum, Kanban and devOps (The new popular brand name for eXtreme Programming). Once you move beyond the team, the organisational context starts to play a bigger and bigger role. This means that as an IT Risk Manager, you need to manage the risk that failure occurs due to context. You need to understand the risk that a practice may work in one context but not in another. In other words, in which contexts is it safe to employ a practice, in which contexts do you need additional tools, and in which contexts is the practice likely to fail.

The Agile Community is an important tool for managing the risk due to the context. The Agile Community is a tightly knit, highly connected group of practitioners who communicate with each other about new ideas and contexts. Anyone can join the community provided they put in the effort to get to know the people involved. Although the strongest relationships are between those who meet face to face at meetups and conferences, there are plenty of on-line communities that are easy to join. The highly connected nature of the Agile community means that it is possible to connect people with a question “What problems did ABC have implementing XYZ” with the people with the answers “At ABC, we did not have a problem because of DEF. Solving DEF is your hardest problem.”. Often the material is in the public domain (e.g. on InfoQ, Vimeo or YouTube) if you know where to find it.

The Agile Community is the repository of all the failure stories that happened before the successful practice was discovered. Having access to these failure stories is incredibly valuable, especially as organisations modify practices and tools in order to make them work in their context. The principles and values help, but sometimes trial and error across a distributed community is the only way to succeed. Its not that the Agile Community hides its failures, its just that a book that says “We tried this and it did not work, we then tried this and it did not work,… and after ten attempts we came up with blah” is much harder to consume than one that says “Do blah, it works”. Think Edison and the light bulb, you get the idea.

The trusted nature of the Agile Community means that members are often aware of failures that are not necessarily in the public domain. One of the filters I apply when recruiting coaches who advocate a certain approach is their attitude to a less well known failure applying that approach. If they are not aware of the failure, they may not be as well connected as they think… and if they are aware of the failure but do not care, they are probably in the community of solutions rather than the community of needs. All coaches need to be in the community of needs, prioritising the organisation that pays them over their desire to promote their pet practice or tool.

The Agile Community is also a fantastic place to seek advice on how to make experiments safe to fail. They do this by sharing the cautionary tales (failure again) and offering feedback on things to watch out for.

High Quality Agile Product and Service Companies employ many highly connected and respected members of the Agile Community. Ronica Roth, Eric Willike and Martin Burns at Rally, Robert Holler and Olav Maassen at Version One, Dennis Stevens and Mike Cottmeyer* at LeadingAgile, Too many to mention at ThoughtWorks and BJSS. These individuals only work for these companies because the companies have a strong Agile pedigree. Their personal reputation and brand means that working for an Agile Industrial Complex company would lead to personal loss of (social) value and effectiveness within the network.

Agile Industrial Complex tend to hire Agile experts on a transactional basis. Often the people they hire have lots of qualifications but they are not trusted members of the Agile Community. This means that the coaches they provide to their clients do not have access to the richness of the shared memory of the Agile Community. This means that context becomes a big risk. Their coaches know the practices and tools, but they do not necessarily know where those practices and tools fail because they have only accessed Agile through the community of solutions (Training Courses). Junior coaches will often work in the Agile Industrial Complex as they build their own experience and personal brand.

Not all of the coaches in an organisation need to be members of the Agile Community, however if none of them are, you are exposing your organisation to context risk.

Thank you to Guy Winterbotham for inspiring this post with his comment yesterday.

*Mike Cottmeyer, what can I say. Highly connected but….remember Agile2009? šŸ˜‰


Cultural Debt

Most of us are aware of the concept of “technical debt”. “Technical debt” refers to short term compromises to speed up delivery of value that have to be paid down in the future. For me, “Technical debt” does not impact quality or value directly, but it has a huge negative impact on lead time and thus massively increases the risk of IT investments. Extending the metaphor, “Cultural Debt” is when we make short term Cultural compromises in order to speed up our Agile transformation which result in significant repayment in the future.

When considering “Cultural Debt”, you need to consider how much harder it will be to bring about change in the future to remove the debt compared to the cost now.

Two common forms of “Cultural Debt” are:

  1. Imposing Agile rather than allowing people to opt-in.
  2. Focusing on Practices and Tools exclusively and ignoring the Principles and Values.

Imposing Agile

Imposing Agile on teams and individuals is completely opposed to the principles and values of Agile. Whilst Agile Industrial Complex members might advise a client to adopt this approach, they are doing their client a major dis-service. The corner stone of Agile is motivated teams who take responsibility for their own actions and make commitments to each other and the organisation. As soon as a manager forces a team to adopt Agile, they not only disempower the team, they also take away their responsibility for their own actions. The team may confirm and adopt the practices being imposed on them but they are less likely to engage and excel in them. As Tony Grout says “Most Agile Practices are the smallest process that will show you what needs to be fixed.” Teams that are not motivated will follow the process by rote and will ignore the improvements they should be making in favour of an easy “box ticking” life. Agile requires self motivated individuals who take responsibility for their own continual improvement. Continual Improvement is less likely to occur in teams where Agile has been imposed and the team are dis-empowered.

Fundamentally the organisation suffers in the long run.

Establishing a team’s responsibility from the start is much easier than trying to get the team to take responsibility later on. The team will have established their own culture and norms and will be reluctant to take on new responsibilities and changes, especially if they have gone through a painful recent change.

Rather than force a team to adopt Agile processes, a more enlightened leadership approach is to place constraints on the teams and let them decide how to meet them. This is what Al-Noor Ramji did at DrKW when he insisted every project should deliver value every three months. The leaders can then provide support for those who want to learn new approach (such as Agile) and tooling that might help them achieve that goal. An even more enlightened leadership might offer the choice of traditional oppressive governance and a light weight IT risk management framework.

Focusing on Practices and Tools exclusively and ignoring the Principles and Values.

The Agile Industrial Complex focus almost entirely on the implementation of new Practices and Tools. That is because it is significantly easier than helping people appreciate Agile’s Principles and Values. Teaching the Practices and Tools ignores that troublesome context stuff.

The Principles and Values are the way organisations adopt Agile in a resilient way. Ignoring the principles and values can result in cargo cult adoption that can fail catastrophically or regress when management attention shifts elsewhere. Much of the value of Agile is in the Principles and Values, even if organisations initially buy the Practices and Tools. Put another way, Organisations miss out on chance to have those super cool productive teams when they fail to acquire the Principles and Values.

However, starting with the Practices and Tools is a great way to build credibility with clients. Starting with the practices can naturally lead to introducing the Principles and Values at a later date. However the Principles and Values should be part of the discussion from the start. It is just that the organisation may not adopt the Principles and Values at the same time as the Practices and Tools.

Implementing the Practices and Tools only is Fake Agile. Sometimes you have to fake it until you make it. But you should never settle for just faking it.

Conclusion

Starting with the Practices and Tools and then adopting the Principles and Values later is an example of Cultural Debt that might naturally be paid down provided the Principles and Values are part of the conversation from the start. Agile coaches should always model the Principles and Values otherwise they will lack credibility later if they try to implement them.

Imposing Agile on teams is an example of Cultural Debt that is very risky from the organisation’s perspective. Forcing teams to adopt Agile may make it difficult or even impossible for the teams to accept responsibility for their own actions later. If your Agile Industrial Complex partner is suggesting you impose Agile, you should show them the door and find a new partner who’s goal is your success rather than an easy life.