Introducing Failure Bias

Most people and organisations suffer from confirmation bias. Confirmation bias prevents us from learning, it means that we seek out information that confirms our belief, and fail to learn as a result. If you or your organisation want to learn, you need to develop a failure bias. A failure bias means that you seek out information that confirms our beliefs are wrong, or have failed. This does not mean we seek to fail but rather we seek the information that we may have failed.

The consequences of the failure bias are:

  1. We actively look for information that refute our beliefs. We need to see failure.
  2. We construct experiments that optimise our learning rather than prove that we are right.

Seeing Failure

Our natural inclination is to seek information that confirms our beliefs. Unfortunately, confirmation bias is a subconscious activity. We are unconsciously competent in the beliefs we trust the most. This means that we must put a great deal of effort into spotting things that refute our beliefs. Even more than that, we must construct strategies that even allow us to perceive the information that refutes our beliefs. This is the heart of the “Break the Model” part of Feature Injection.

Optimising Learning

Claude Shannon’s information theory tells is that learning is greatest when uncertainty is greatest. We learn the most when the chances of success are 50/50. When developing products we have two phases, the first is to learn about our customers, and the second is to exploit that learning. During the learning phase, the experiments should be focused on learning (50/50) rather than confirming our belief. This is a huge challenge in risk averse cultures where failure is not tolerated. Executives in risk averse cultures should focus to ensure that there is a healthy balance of failure and success. During the learning phase they should ensure that confirmation bias does not obliterate learning. Given the nature of risk averse cultures, they should hold up a lack of failure during the learning phase as the worst kind of failure.

The failure bias is what separates startups from traditional organisations. It also separates those that will survive from those that will not.


The Executive Learning Trap

In 2004 a colleague helped me try to create a learning organisation. My colleague had studied learning organisations for his MBA thesis. I have never forgotten his counsel…

“The single most important factor to the success of a learning organisation is management’s attitude to failure”.

Failure and its relationship to learning as individuals and organisations is well established with popular books like “Black Box Thinking” by Matthew Syed providing an accessible Gladwell style summary* of the subject.

The unspoken assumption is that Executive’s do not understand the relationship between failure and learning. That organisation fail to learn because executive do not understand the importance of “safe to fail” experiments and as a result do not create environments that allow them.

Over the years I have worked with a number of senior managers and executives who do get it… Who are supportive, but still the learning and the failing fail to happen.

So I am going to operate using a new set of hypotheses to see if I can create the elusive organisation learning:

  1. Most executives are unconsciously competent learners.
  2. Most executives do not have a theory of learning and coaching.

Expanding on this:

  1. Most executives are excellent learners. Quite often they have an experiential learning style where they construct safe to fail experiments to find solutions to problems they have not encountered before. This experiential learning style means that they often use strategies that they have not studied but rather developed using their own knowledge and experience. An anthropologist may use a concept from anthropology to solve a problem in organisational design. In effect, they have exapted a concept from their own field of study rather than adopt a standard Harvard Business Review approach. This makes it hard for them to communicate their approaches to their organisation as they do not have material for them to refer to.
  2. Although executives are unconsciously competent at learning by creating safe to fail experiments to learn, they do not have a theory or model that they can use to communicate to their organisation. Although they know how to learn themselves, they do not know how to coach their organisations to learn, and more importantly they do not know how to coach their organisations to manage the risk around learning and ensure that all experiments are safe to fail.

The implication of this shift is that we should not encourage executives to create environments that promote learning by tolerating failure. Instead we should coach executives so that they transition back to conscious competence by building a theory of learning that they can use to coach their organisations.

Its a hypothesis as I need to try a new approach. I’d love to hear from anyone who is ahead of me in this journey.

* Check out Dave Snowden’s Lean Agile Scotland keynote covering Pseudo Science et al.


Why Building the Wrong Thing is so so easy

One of the key risks that organisations need to address, is building the wrong thing. No one willingly builds the wrong thing, so why do Product Owners build the wrong thing?

The first question to understand is “What is the wrong thing?”. The wrong thing is building anything other than the right thing?

So the second question is “What is the right thing?”. The right thing is a subjective decision based on the context of the organisation and the needs of the customer. At least Product Owners always prioritize the building of what they think their customers or organisations need. Or do they?

There is a nasty cognitive bias built into most organisations that Executives need to actively manage. That bias is your architecture and organisation structure. Everyone knows that it is easier to deliver something if only your team is involved. They know that involving another team in their department is harder so will try to avoid doing that. They know that involving another team in another department or division makes it even harder, almost impossible in fact. So as a result the way that many product owners approach the problem is not to ask “What does the customer need?”, they ask “What does the customer need, that I can do with my team or another team in my department?” As such, the Product Owner is more focused on their capability than the needs of the customer. They will (often unconsciously due to the culture) pre-filter those things they know are hard to deliver.

So the answer is simple, the reason why building the wrong thing is so so easy, is because building the right thing is hard.

Some Agilistas around the world have attempted to avoid this problem by creating “autonomous product” groups such as Spotify’s Tribal Model or SAFE’s Agile Release Train. My experience is that no matter how hard you try to architect your product solution to remove dependencies, there is always a customer or organisational need that crawls out of the woodwork that requires work by two or more groups. There is always a new regulatory regime, or a new way of servicing the client, or a new operating system that requires coordinated changes across multiple “autonomous product” groups.

So instead of trying to build autonomous product groups, or constructing projects from scratch, lets consider a middle ground. What would that look like?

  1. Create long lived teams based on a carefully (and continuously) thought out but dynamic architecture. Smaller components supported by a handful of teams are more stable than massive chunks of the business.
  2. Use Quarterly Capacity Planning to optimise the delivery of value from the portfolio, and make it easy to pull together teams needed to deliver the customers needs. Ensure that all of the business have an agreed ordered backlog.
  3. Understand that pulling together a group of teams who do not normally work together is constructing a value stream from scratch. Get the teams to collaborate and coordinate using “scrum of scrum” stand ups and regular retrospectives (rather than plan up front). Assign responsible to facilitate the collaboration and coordination to an individual for each piece of work. Make it easy to reassemble groups of teams by doing it frequently as required by the work rather than try to avoid it.
  4. Build what the customer and organisation need rather than build what is easy.

This was the approach Skype took to managing the risk of building the wrong thing. We may not have built the right thing every time, but at least we did not build the wrong thing because it was easier than building the right thing.

Its another genius thing worth studying that Mark Gillett created.


All Estimates are Waste, some are useful

Estimates are crack cocaine for managers and executives. Estimates are an emotional crutch that give managers an illusion of control over an inherently risky process. As well as being an expensive waste, estimates can drive unhealthy behaviour in teams. The problem with estimates is that there are different forms of estimate that are appropriate for different decisions and different levels of understanding, and unfortunately managers do not always use the appropriate estimates.

To begin with, an estimate is a “tea bag”, it is part of the solution needed to create an output for the manager. The output is required to satisfy a need for the manager. There are two outputs from an estimation process:

  1. How much will it cost?
  2. When will it finish?

In actual fact, you need to answer the second question before you answer the first.

The cost of an IT investment is the duration of the investment multiplied the day rate of the investment. Consider a team of eight developers who spend half of their time on support and half on new investments. In addition to the developers are a product owner, UX, managers, etc. amounting to a further sixteen head count on the project. If the investment is estimated to run for a further thirty days, then the cost will be 30 * 20 person days = 600 person days. i.e. All the people involved in the investment rather than just the four developers. That is why an understanding of Theory of Constraints is so important to managers, especially those who have been trained in cost optimisation which tends to create constraints and increase costs.

So the cost of the based on the duration of the investment, or rather, when will it finish. How do we determine that? There are two measures we can use:

  1. Count of stories
  2. Story Points

According to the work of Todd Little and Troy Magennis, there is little extra accuracy using story points over the count of stories. However there is a real advantage of the story count as it is available much earlier in the process than the story points. A reasonable estimate of the count of stories is normally available very early on in the process. Story point estimates are normally available about a sprint before the story is developed. That is, after the product owner has worked with the UX and Data dudes in the Product Owner Squad to create the prototypes, test them with users, write the gherkin format acceptance criteria and then slice the epic into stories that can be estimated by the team in the product refinement session. The alternative is to create a whole bunch of pure waste to create faux story point estimates much earlier than they should be available. Faux story point estimates that are not following the normal process and are thus not really story point estimates, and provide a false level of comfort.

Estimating when an investment will finish is best done by looking at a Cumulative Flow Diagram and projecting based on the rate stories are being created and completed. Consider the following Cumulative Flow Diagram. The black dotted line is “now” and the red dotted line is when the investment is due to finish. The red dotted line is calculated by projecting the rate at which stories will be created and finished. (Troy Magennis’s “Cool and Fabulous Spreadsheet” does something similar with numbers instead lines, and in a much more grown up way using statistical confidence intervals).

screen-shot-2016-12-28-at-11-49-27

A week later, the manager looks at the CFD and sees that the estimated finish date has moved because the rate at which the stories have completed has increased. This may be because the amount of support work has reduced due to higher code quality, or because the team are improving and going faster. The manager can investigate and potentially identify changes to the process to improve the estimate of the number of stories.

screen-shot-2016-12-28-at-11-49-41

A week later (in an alternate reality), the manager looks at the CFD and sees that the estimated finish date has moved because the number of stories has increased. This may be due to unforeseen functionality, or because a story was more complicated than expected and needed to be broken down. The manager can investigate and potentially identify changes to the process to improve the rate of delivery of stories.

screen-shot-2016-12-28-at-11-51-12

A week later (in yet alternate reality), the manager looks at the CFD and sees that the estimated finish date has not moved because the number of stories has increased but that has been offset by an increase in the rate that stories are completed.

screen-shot-2016-12-28-at-12-05-20

As well as understanding that there is a change in the estimated finish date, the manager can understand the source of the uncertainty that has created the change.

Now lets consider the same graphs drawn using story points. To begin with, we need to get some “experts” in room to estimate all of the stories, because we would not want to get the whole team to give an estimate as that costs too much. That means we have guess what the stories will be (before user testing and all that). Our “experts” will only be too pleased to provide a story point estimate, even though it is not a story point estimate. It is an expert “estimate” rather than one by the team doing the work, and it is not based on a “Definition of Ready” story, its a faux story point estimate. This means at some point during product refinement, the estimate will change to a real story point estimate and we will have some stories estimated using story points and some using faux story points. This mixture of story point methods is misleading and as a result, the manager will pressure the product owner to complete all the stories to a state of “definition of ready” so that they can get a reasonable estimate. There is no sensible* intervention that a manager can do to improve the quality of the estimates.

( * We could adopt a waterfall approach to creating stories. In order to improve the estimates, we could increase the risk and cost to the organisation substantially which isn’t really sensible ).

Even worse, there are now three reasons for a change in the estimated finish date, as well as the number of stories created and completed, we now need a third dimension for the change in story point estimates. In other words, to understand what has happened, the manager needs to study two graphs** and the difference between them. What was obvious on the CFD, has now become complicated to understand and needs an “expert” consultant to explain. This lack of transparency leads the manager to fail the Agile Risk Framework Audit that requires them to know and understand what is going on in their investment.

( ** I tried to think how you would represent in a useful way this but gave up as it was too complicated, and it was silly anyway)

One day, the manager has a revelation! Instead of using faux story point stuff to estimate the finish date (and hence cost), lets encourage the product owners to write stories of a broadly similar size. From now on, the largest story allowed will be an “eight” (or a “five” for teams with one week sprints). No more stupidly massive “thirteen” or “twenty” stories. That will mean I can get my estimates for free without any stupid and wasteful “faux story point” estimation sessions.

So if you consider yourself a competent manager, go read Todd Little and Troy Magennis stuff on estimation and prediction.

Todd Little Key Papers

 

 


Simple OVER Complicated

Mark Twain famously wrote “I didn’t have time to write a short letter, so I wrote a long one instead.

A more contemporary version might be “I didn’t have time to write a simple depiction of Agile, so I wrote a complicated one instead.

A more truthful contemporary version be “I wanted you to think of me an expert so I created a complicated depiction of Agile, especially as I did not have enough experience to create a simple one to handle the complexity of the context.”

This might explain the SAFE diagram and thw Deloitte Agile diagram.

Simple in Practice

Simple is vital for adopting Agile at the organisational level. Tony Grout, Lisa Long, myself and many others spent significant effort at Skype implementing and supporting a process to coordinate two hundred product owners and three hundred development teams.

Initially people struggled with just understanding the process. Then once they understood it, there was a desire to improve the process. This usually involved adding to the process, making it more complicated.

Tony, Lisa and I had a single design principle to guide us. The process should be so simple that no one should be able to use the process as an excuse for not following it. Anyone not following the process would look foolish if they failed to do it properly. This was because we knew there might be people who opposed the introduction of the process.

Opposing complicated-ness was difficult and we had to be constantly vigilant. For eighteen months, the process evolved. Each retrospective, the group would say the best thing was the “preparation” and the thing they most wanted to improve was the “preparation”. The initial meetings took three days with approx sixty people face to face in Tallinn, London and Redmond. The last meeting I’m aware of involved about twenty people on a conference call for two hours. Much of the quarterly meeting was moved to the “preparation” activity. We had started trying to create a week by week plan for the quarter with everyone in the room. We dropped the planning and focused on creating an ordered backlog based on the available capacity in the teams.

Although the tight knit team running the process increased the level of automation, the product owner team had only two simple instructions:

  1. To get an estimate from the product owners of all parts of the organisation needed to help them deliver their initiative. This meant as a minimum they had to have a conversation with the other product owner. We just did not specify what the conversation covered. We gave guidance that the conversation should last about ten minutes, and the product owner providing the estimate might want to discuss it with the tech lead and test lead, but not the team. We faced a constant challenge with teams wanting to provide ever more accurate estimates.
  2. They had to create an ordered backlog based on the capacity available to the teams. We did not tell them how to come up with the ordered backlog, only that they had to come up with one.

The success of the process was based on a simple process that two hundred product owners followed.

The key point is that there is a natural tendency towards complicated and it requires effort to keep things simple.

Simple in Communication

Simple communication requires practice and preparation. Communication goes through three stages, excitement, boredom and effectiveness.

  1. Excitement – During the excitement phase, we love a new idea and every time we explain it, we embellish the explanation with extra details and anecdotes. Our description of the idea becomes more and more complicated and we feel the need to add more and more “important” details.
  2. Boredom – After we have explained the idea a number of times, we get bored of it and start to drop the “important” embellishments. Our boredom with the idea leads us to drop truly important details to the explanation which results in confusion and additional effort. We learn what is genuinely important to our explanation which cannot be dropped, and what is unimportant and can be dropped. Our laziness leads us to the perfect explanation.
  3. Effectiveness – Eventually we reach the point where our laziness has led us to an explanation that is as simple and short as possible.

This process requires the need to practice an explanation many time. The best way to practice is one on one. People are less likely to ask clarifying questions in a group, and they are also more likely to ask challenging questions. You need to practice with people who do not know the material, and people who know it well… Both groups will give different but valuable feedback.

So aim for simple, and understand that complicated is a sign of a flawed understanding. As Einstein said “Everything should be as simple as possible, but not simpler”.


The Executives New Clothes

There is no reality. Everything is an abstraction. Your reality depends on which abstraction you choose to believe in. The decisions you make depends on your reality. The decisions lead to your behaviour. Changing the way you look at the world, changing your phenomenology can be a scary thing. In order for a coach to help a manager learn a new way of looking at reality, they must first build a relationship of trust with that manager. The trust is necessary so that the manager will trust the coach will be able to get them back to safety.

Lets take a look at a small example of this. One of the main ways that senior managers look at projects is the status report. The status report is the way that projects make themselves visible. Status reports are one of the big differences between traditional and Agile projects. Traditional projects present a static view of status whereas Agile projects present present a dynamic or graphical view of progress, lets consider the first derivative or “velocity”.

A traditional status report might say that 90% of stories are complete. This sounds pretty good doesn’t it? However, lets look at the dynamic view of progress using cumulative flow diagrams (CFD). Consider two different “Epics” where 90% of stories are complete.

In the first CFD, risk averse development team, the team have completed the easiest, least risky stories first. They are left with the hardest riskiest stories and it is impossible to predict when the last 10% will be completed.

screen-shot-2016-12-23-at-08-33-39

In the second CFD, risk managed development team, the team have completed the hardest riskiest stories first. They are left with theĀ easiest least risky stories and it is easy to predict when the last 10% will be completed.

screen-shot-2016-12-23-at-08-36-38

The point is that a static view of 90% complete hides the fact that the project could be in a really good place, or in a really bad place. Revealing this to a manager has no real up side for them but may demonstrate that a “successful” project is actually in a really poor state. Viewing the poor state for the first time this way, there are a number of ways that the manager and the culture they operate within may react.

  1. They accept the new reality.
  2. The reject the new reality.

From the coaches perspective, they are hoping for option one. This is only likely to happen if the manager is confident that they can survive the revelation to their management. The ability to accept a new reality is based on how safe they feel accepting the new reality. That is either because they have the options they need to survive and thrive in the new reality, or the culture they operate in is accepting of failure, and the discovery of new contexts. The new reality may be an opportunity for the manager if they are able to cope with it, and especially if their competition in the organisation is unable to.

In the event that the manager does not have a solution to the situation, they are going to have to depend on the coach to help them out… And that is only going to happen if they trust the coach. If they do not trust the coach and their ability to help them, they may reject the new reality, and the coach at the same time. They may attempt to hide the new reality from their stakeholders and fix the problem with their existing trusted colleagues.

Introducing a new reality into an organisation is risky to the individuals who experience the new reality, especially if they do not have the options to handle what they perceive. The person showing them the new reality is equally at risk if they do not have a trusted relationship with them. However the “new” reality has always been there. The executive has been wearing new their new clothes, they have been in the altogether the whole time. The difference is they are now aware of the situation and can potential react with new options.

As Dan North puts it “You go into a darkened room and turn on the lights, there are four tigers in the room. There were always four tigers in the room, its just you could not see them until now.”

Please don’t kill your coach because they switched on the lights.


The hedgehog, the fox and the CIO.

I was chatting* to Tony Grout yesterday and he introduced me to the Hedgehog and the Fox concept… The fox knows many things, but the hedgehog knows one important thing. The IT Risk Management Framework is a fox thing, it is a super clever subtly complex framework for clever foxes to provide real governance and control over IT investments, change culture, and keep clever foxes happy, all at the same time. Its just tricky for CIOs who are managing “Hedgehog” organisations who can only know one important thing.

Agile is a fox strategy which attempts to wrap up decades of good practice into a single single Hedgehog term. The only problem is when you try to execute on the Agile hedgehog strategy you discover the fox strategy is hidden inside it. Even worse, you discover that no one can agree on what Agile is. Is it SAFE or LESS or the DADDY? Oh no, Its not even a fox strategy, you just unwrapped a whole skulk** of fox strategies all fighting and nipping at each other.

Instead, lets look at the IT Risk Management Framework which consists of four drivers:

  • Deliver Value
  • Sustainable Quality
  • Reduce Lead Time
  • Manage Risk

Lets consider each one at a time.

  • Deliver Value is really a business concern. If IT investments are not delivering value, the CIO can offer advise but fundamentally they are going to be politely shown the door back to the IT department ghetto if they try and take control. The business will handle this, and the CEO will own the definition of value.
  • Sustainable Quality is a must. If the products do not have sustainable quality from the customer’s perspective, the customers will leave. Once again, the business will handle this.
  • Reduce Lead Time is under the control of the CIO. By control, I mean responsible, and they can rightly be held accountable by the CEO for this key metric. Furthermore, they tend to own all the resources to address the significant parts of the value stream.
  • Manage Risk is best done by having short lead time, and with instigating a culture of transparency. So Reduce Lead Time gives you a big chunk of Manage Risk as well.

So in summary, a CIO’s hedgehog strategy is Reduce Lead Time because it gives you most of Manage Risk, and the business leadership will cover Deliver Value and Sustainable Quality. Something that Andrew Green, Al-Noor Ramji, JP Rangaswami and Lee Nicholls have been doing for years.

Merry Christmas CIOs, Reduce (Weighted) Lead Time becomes your measurable transformation strategy.

P.S. Obviously Lead Time is not an effective metric as it cannot be scaled, and hence we use Weighted Lead Time instead.

* OK, so this is mostly Tony’s stuff but I’m gonna write it down because he most likely will not.

** Who knew, the collective nouns for foxes are Leash, Skulk, Earth, Lead, Troop!