Author Archives: theitriskmanager

About theitriskmanager

Currently an “engineering performance coach” because “transformation” and “Agile” are now toxic. In the past, “Transformation lead”, “Agile Coach”, “Programme Manager”, “Project Manager”, “Business Analyst”, and “Developer”. Did some stuff with the Agile Community. Put the “Given” into “Given-When-Then”. Discovered “Real Options”

Balancing the portfolio using Cynefin

The Cynefin framework can be used to assess whether the current and planned portfolio are balanced appropriately.


There are three patterns of portfolio investment depending on the maturity of the product.

  1. A startup would have a portfolio dominated by investments in the Complex domain as the organisation strives to understand customer needs and whether there is a viable product.
  2. A teenage organisation would see the majority of investment shift from the complex to the complicated domain as it seeks to scale and exploit its knowledge of customer’s needs.
  3. A mature organisation would have a portfolio dominated by investments in the complicated domain with a healthy slice of investment in the complex domain providing knowledge for the next generation of investments in the complicated domain. These investments in the complex domain allow the organisation to better understand customer needs as they evolve. Although an organisation should have a portfolio where the majority of investment is in the complicated (and obvious) domain, a portfolio with no investments in the complex domain will probably lead to the organisation losing touch with its customers and driving off the cliff.

Investments in the chaotic and obvious domains will normally be significantly smaller than investments in the complex and complicated domains. In times of crisis, investments in the chaotic and obvious domain may dominate.

  1. When the company loses touch with customer needs, investments in the chaotic domain may dominate. In such situations, normally heralded by a significant increase in churn, the organisation will be forced to focus a disproportionate amount of investment into addressing the issue. In this situation, the portfolio will naturally shift back to a healthy balance.
  2. In times of crisis, when the industry is forced to change by regulators, investments in the obvious domain may dominate. The Chief Product Owner (CPO) must ensure that the portfolio returns to a healthy balance and “crisis investment” does not dominate beyond the crisis. Managers responsible for Investments in the obvious domain tend to have a “Just Do IT” / command and control attitude. These managers “know what is needed” and do have regard for those who want to understand customer needs. The CPO should ensure that valuable people who research and understand customers are not lost during the crisis. These kind of managers find it hard to give up the significant resources at their command after the crisis, and the transition back to a balanced portfolio will require strong leadership.

Investments are in the disorder domain when the product organisation may not agree on which domain some of the investments  are in.

  1. “Four corners contextualisation” can be used to better understand the investment and how it should be treated.

Automated Test Coverage as a goal is at best, misguided

Setting Automated Test Coverage as a goal is at best, misguided. Automated test coverage is useful as a strategy or as a diagnostic metric, however using it as a goal is idiotic and will lead to waste and the wrong behaviour.


For any IT system, there are three options for testing:

  1. Automated tests
  2. Manual tests
  3. No tests

Lets pop the why stack on automated tests. Automated tests are faster and more reliable than manual tests. Automated and Manual tests are normally safer than no testing. So the reasons for automated tests are:

  • Reduced lead time.
  • Reduced variability in lead time.
  • Lower probability of a production incident.

Our goal should be to improve one of these metrics, normally reduce lead time. Lead time and automated test coverage are correlated. If you attempt to reduce lead time, one of the strategies you are likely to apply is to increase automated test coverage. As such automated test coverage is an excellent diagnostic metric to help the team identify ways to reduce their lead time.

There is not a causal relationship between automated test coverage and lead time. Increasing automated test coverage does not automatically reduce lead time. Many years ago I worked on system with no automated test coverage. Management imposed a 100% test coverage goal for all systems. Everyone on the project stopped working on anything else and spent a few days writing tests. As the business analyst I was given a list of classes and told how to write standards tests for each method to ensure the test coverage tool would register us as meeting our 100% target. We achieved 100% automated test coverage but no improvement in lead time or anything. The activity generated no benefit to the organisation, it was pure waste.

If you set reducing lead time as a goal, you will likely see an increase in automated test coverage. If you set increased automated test coverage, it is possible you will see no benefit.

Investing with Cynefin: Disorder

Disorder is the fifth quadrant in the Cynefin model. Disorder is where there is no clarity about which of the other domains should apply.

“Here, multiple perspectives jostle for prominence, factional leaders argue with one another, and cacophony rules”.


The solution is “obvious”, bring the leaders together to perform a “complicated” ritual that reveals the “complex” nature of the problem and hope that the personalities involved do not turn the conversation into something “chaotic”.

A technique like four corners contextualisation can be used to facilitate a conversation between decision makers. The discussion will normally reveal that the problem is one of granularity. Decision making is a classic example of something that is often in the domain of disorder. However, when we break it down into a lower level of granularity, we discover that it falls in the other four domains.

  • The output, an ordered list is obvious.
  • The process such as cost of delay is complicated.
  • The interaction between the investment options, the available resources and the participants are complex.
  • The behaviour of warring factions is chaotic.

By moving to a lower level of granulation, the domains become apparent.

Investments that remain in the disorder quadrant indicate a dysfunctional decision making group. Often a hippo will force these items into one of the other domains. Disorder is often a symptom of a group stuck in “storming” that is not coming together to communicate.

Investing with Cynefin: Complicated

The complicated domain is the realm of strategic superiority. Organisations know with certainty how customers will behave, however it is not common knowledge. This is where organisations should focus their strategic investment.


The complicated domain is where organisations should be making larger investments as this is where they have competitive advantage. The only place where organisations should be making larger investments is where they are forced to due to regulatory dictate (Obvious) or to resist the vertiginous draw of the cliff (The Cliff). The complicated domain is where constrained resources should normally be deployed.

The complex and complicated domains are not binary in nature but rather “linear” with 0% certainty at one extreme and 100% at the other. As such, the investments are best managed using the Kelly criterion. The nature of the experiments change. Whereas in the complex domain, the experiments relate to understanding needs, in the complicated domain, the eperiments relate to the scope of the needs.

For organisations, the danger with the complicated domain is that too many investments are made in it. Either because they are classified incorrectly due to perverse cultural incentives or because the organisation is utterly risk averse. One is reminded of the risk averse anthem “No one ever got sacked for investing in the complicated domain. buying IBM.”. The real message being that perhaps some people should have been sacked for failing to think for themselves.

In summary, investing in the complicated domain is the easy option. Therefore the investment decision process should make it difficult to do so.

Investing with Cynefin: The cliff

In the Cynefin framework there is a cliff between the Obvious and Chaotic domains. Systems fall down this from the ordered domain down into the Chaotic domain. This is eloquently summarised by Mark Twain…

It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.


Retail businesses on the high street are currently thrashing around at the bottom of this cliff. After years of pulling levers to control shoppers and extract profits, the bricks and mortar retailers now resemble Dr Who randomly playing with the controls of the Tardis as it bounces around reality. The reason is that the internet has changed the fitness landscape of business and new apex predators have emerged that are disrupting existing players.

When organisations find themselves inside the OODA loop of an Apex Predator in an unfamiliar fitness landscape, they have to act, sense and respond. As a result, investments in the chaotic domain may result in disrupting the investment process in all of the other domains. Although investments in the chaotic domain should ideally be small, there are occasions when all the resources of the organisation will need to be focused on them.

Ideally organisations should avoid the cliff all together, and if necessary they will need to invest all of their resources to keep themselves away from it. Organisations can use metrics to detect the presence of the cliff. In particular they can use churn metrics. An organisation would typically have three sets of customer metrics…. Number of customers, customer activity and customer revenue. The trend is more important than the actual value for investment decision making.


In the example above the metrics may initially appear healthy. Every week sees a 10% increase (Green). However looking at the churn number indicates a problem occurred in week 4. At this point, the organisation needs to act as it heads towards the cliff / lands at the bottom of it. The action required is normally to gather information. Why has churn just jumped up? Which customer need is not being met, or possibly which customer need is being better met by a competitor’s product or service? The research will either involve data analysis or user experience research (to understand user needs/jobs to be done).

Many organisations wallowing at the bottom of the cliff have no data analysis or user experience research capability. For these organisations, the ‘act’ is simple, to acquire these capabilities.

Modern organisations wishing to avoid the cliff need to invest in data analysis and user experience research capabilities before they find themselves being disrupted by another organisation’s OODA loop.

Investing with Cynefin: Complex

For investments, the complex domain may be defined by the strategy used to manage it “Multiple hypotheses tested using safe to fail experiments”. The complex domain by definition contains uncertainty (risk) and is about acquiring knowledge. Investments the complex domain should be the smallest safe to fail experiment that test a hypothesis. Organisations should not be making investments that are large or that are not safe to fail.


Investments involving technology have three categories of risk¹ :

  • Delivery risk.
  • Business Case Risk.
  • Risk of damaging the existing business model.

The Kelly criterion can be used to help understand the maximum size of investment in the Complex domain. Fundamentally investments in the complex domain are about reducing risk (uncertainty) by acquiring knowledge (certainty). The first two categories of risk are about failing to achieve an upside, they are not about protecting against a down side. Managing the risk of failing to achieve an upside is done by ensuring that individual investments are not so large that they damage the portfolio. Hence the Kelly criterion can be used. Whilst Kelly can assist with the first two categories of risk, alternate strategies are needed to protect against a down side.

Managing the risk of damage to the existing business model.

Protecting against down side is about managing the risk of unintended consequences. If a consequence is known, then a specific option should be created to ensure that the investment is safe to fail. Therefore ensuring investments are safe to fail, requires the investor to have options to detect problems and return the system to safety. This means the following:

  1. Effective monitoring is required to detect unintended consequences.
  2. Options to return the system to safety.
  3. Failure containment.

Failure to monitor for unintended consequences is an abdication of responsibility and normally indicates a risk averse culture dominated by Hippos. The Hippos either accept of ignore risks that might occur.

Time is the key element of options. If the time the system can survive is less than the time it takes to return the system to safety, more options are required before the investment should be considered safe to fail.

Finally, one of the key differentiators between contemporary organisations like and Google, Facebook and Netflix and traditional organisations is that contemporary organisations manage risk rather than ignore it. They create failure containment.Contemporary organisations roll out investments to customers gradually. They test hypotheses to ensure that not only does the investment work, but also the customers have the anticipated, or at least a beneficial, behaviour change. The idea that you would roll out an investment to 100% of your customer all at once is the equivalent of putting all of your money on “red” at the roulette table… You are not investing, you are gambling. You might get away with it a few times but eventually you will do a “Knight Capital”.

The Complex domain is fundamentally about risk management. As such, it is the domain where Real Options are most effective.

1-Original article by Steve Freeman and Chris Matts. Published in Agile Times, 2005

Investing with Cynefin: Obvious

The counter-intuitive aspect of investments in the obvious domain is that organisations should seek to minimise investment in this domain. As Dave Snowden once pointed out, organisations have no competitive advantage in the obvious domain. Investments in the obvious domain are often as a result of constraints imposed by regulatory bodies.


Many organisations have a special categorisation for investments “Regulatory and Mandatory”. These investments are often referred to as

“Must be done!”

A more intelligent description would be:

“Must be done if the organisation wishes to remain in that particular business.”

The reframing is very important. In the late naughties, an investment bank faced with expensive regulatory investments decided to sell its commodities business. This freed up capital that could be invested to make its other business lines stronger.

A common mistake when investing in regulatory and mandatory investments to to try and be the best, to attempt to invest and excel. Using Niel Nickolaisen’s purpose alignment model (Check out Kent McDonald’s excellent description here along with other useful tools), regulatory investments fall in the parity quadrant. Investors should seek to be “good enough” but not invest and excel. This often means implementing a third party solution if an appropriate one is available. Where an appropriate third party solution is not available, the solution should be architected in such a manner that it is easy to migrate to one when it is available.

The strategy for regulatory investments is to minimise total cost of ownership with a “good enough” solution. Unfortunately organisations often misinterpret this as “Implement with your cheapest resources” which is a path to failure and excessive costs. Given that regulations in a market normally increase and are rarely removed, the organisation should consider the long term implications of any solution. This means that regulatory investments should be implemented using eXtreme programming techniques that support safe, rapid and cheap modification in the future.

Realistically, the only way to turn regulatory investments into strategic investments is to deliver a solution to the regulator before your competitors. That way, your organisation can influence the regulators and disrupt any competitors using traditional techniques.

In summary, even though the investment may be obvious, the solution may require careful thought.