Strip Maps and Risk

These days if we want to get from one place to another we have a number of choices available to use. When driving in the car, I tend to use SatNav. I use SatNav even for familiar routes because it is fairly reliable at rerouting me when poor traffic conditions block the way ahead. Just this weekend I went on my weekly pilgrimage to see my children, and SatNav took me off the motorway to partially avoid a traffic hold up. Sometimes my SatNav throws a wobbly. There are a few points in my regular journey where I have come to learn (at my previous cost) that I should ignore the SatNav. These are normally because the road has changed since my SatNav was installed (and the map is not updated). Without the traffic updates, I probably wouldn’t bother with SatNav at all most of the time. I prefer to check out my route in advance using google maps, and use the map on my phone when I get close to the destination.

Before electronic maps we had the familiar A-Z Atlas in the car. Everyone I knew had easy access to an A-Z of London. You had to update it every couple of years but in reality, we would get the tube to the closest point and follow directions from there.

Even in the world of SatNav and perfect maps, we have to exercise caution. Even in the world of perfect maps, the maps don’t tell us everything. Maps don’t tell us which routes are safe or dangerous, whatever that danger might be. Trusting too much to a map is dangerous. In cities like London, with its twisty roads formed by generations of cows and sheep herders, it is easy to get lost and ending up pointing in completely the wrong direction. Even with a map, people in towns may find a compass useful sometimes.

Before comprehensive maps, people used strip maps. Particularly in Medieval times, these maps were used to help people travel along popular route such as pilgrimage trails. This is an example of a strip map.

Medieval Strip Map

If you were going from London to Jerusalem, the strip map was pretty helpful. But what if you were starting out in Paris, or Stockholm, or Beijing? Then the strip map was of limited use. You may have another strip map that converged with a point on the Strip map to Jerusalem but you have no guarantee that its the safest or easiest journey from where you are starting out.

Last week I discovered the strip map metaphor to be particularly useful to explain to people about Scaling Agile. Rather than tell them we have a SatNav that will get them to Jerusalem, I explained that we were on a journey that few if any had ever been on before. There were Strip Maps (experienced individuals and experience reports) that could help us with parts of our journey but we had to be aware that we may come across obstacles that others have not described yet. In Cynefin terms, it activated people to a heightened state of awareness so that they would challenge the route rather than simply follow it without thinking. For team level Agile we have SatNav supported in five dimensional cyberspace. For organisational level Agile, we have a few strip maps written on vellum.

For Scaled Agile, we are still mapping out the territory. In fact, we are still determining the dimensions of the territory. Culture is one dimension, as is the Cynefin domain. Scale changes everything so that’s probably going to end up as a dimension as well. As will the business domain. As the saying goes, if you want to get to Jerusalem, I wouldn’t start here.

There are some successful strip maps. If you want to get from Croydon, New Jersey to Jerusalem you can follow a Safe route. In Cynefin terms, if your company is in the over constrained “Obvious” domain, then Safe will help you migrate from DSDM or RUP to the true “there is only one way” “Obvious” SAFE route to the Jerusalem Hotel in Nevada just a few mines south of the place that features at the end of “Thelma and Louise”. The danger is if you happen to live in Meinehatten, Germany, following a SAFE route might lead to more danger than finding a local explorer with experience of a similar journey to Masada. Make sure the explorer has an extensive network of fellow explorers to reduce the risk.

We are also aware that following a strip map blindly can lead to real pain and heart-ache. There are whispered rumours of a Leming Organisation following a religious visionary over a cliff (I prefer to think of that visionary as the blind man in “The Life of Brian” who was blind but now can seeeeee.)

Strip maps are a great metaphor for the state of Scaling Agile. Strip maps encourage a “Safe Fail” OVER “Fail Safe” mentality. Before we move forward, lets make sure we can get back to safety. As a metaphor it helps organisations understand that they will need lots of strip maps. They will also need compasses, and sextants, and models and beliefs (theory), and they will also need climbing ropes for those experiments that are not “Safe to Fail” enough.

My gratitude to Martin Burns for introducing me to strip map metaphor after I wrote about “Scaling Agile being off the map”.


Understanding Uncertainty’s impact on Learning.

In Commitment, Olav and I wrote that the “rational” order of preference is:

  1. Right
  2. Uncertain
  3. Wrong

We also wrote that the observed preference for most people is:

  1. Right
  2. Wrong
  3. Uncertain

People would prefer to be wrong than uncertain. We know this because they make commitments earlier than they should, and destroy options unnecessarily. A preference for uncertainty would result in more learning. A preference for learning would be an advantage within a “Community of Need”. An aversion to uncertainty is an advantage in a “Community of Solutions” as you can be more compelling and forceful in your argument about a solution. You can be more certain.

We learn something new when we perceive it has value for us.

consciouscompetence2

It is stressful to be aware of things that are valuable that we do not know. If we crave certainty, we ignore the value of these things. We can express value using this formula:

valfiltinfo

We can considering this value in Kolb’s “Circle of Learning” ( a.k.a. Feature Injection’s “Break the Model” ).

kolb

“Spot the value” means we spot an observation, Ix ,such that the value Vx, calculated using our value model

V = F ( Io … In, Ix )

is different to the value observed. In effect, we have an arbitrage situation. The difference between the observed value and the calculated value are different. We should act accordingly, learning and applying the new idea.

This seems fairly straight forward. It becomes tricky because our value function F ( Io … In ) acts as a filter on our perception of reality. The more certain we are, the less likely we are to differences between two things that reveal a problem with our value function. The more we are comfortable with uncertainty, the more like we are to identify differences.

This is very important to understand the impact of uncertainty on learning. If we are certain, we will not notice the problems. Its not a case of we ignore then, we simply will not perceive them as important. In fact, its when we notice these observations that do not fit our model that we are at danger of shifting from “Obvious” where we are certain, to “Chaos” where we see lots of observations that do not fit our model and the model effectively collapses as it is no longer of use.

Knowledge Options

Learning new things takes a lot of effort. Twenty hours of concentrated practiced has been suggested as a minimum to become barely sufficient in a subject. More effort is required to become proficient and significant effort is required to master a subject. “Community of Solutions” value masters of a subject. They will often be proficient to a level where they can make a tool work even though it is not a natural fit for the context. People in a “Community of Needs” need to understand the value of an idea, the context in which it is best applied. They also need knowledge options. A knowledge option involves knowing the value of an idea and having an option to acquire proficiency in the skill within a certain timescale. You can learn more about knowledge options in the “The Lazy Learner” talk. The most powerful option is to be a member in many “Community of Needs” with access to practitioners who can guide and help you learn quickly.

Organisations hiring Agile Coaches should always hire people from the “Community of Needs”. Trainers can come from either community. If you hire a coach from the “Community of Solutions”, they will attempt to solve your problem with their favourite solution, regardless of whether it is the best solution. They will attempt to demonstrate it will work in all contexts, even if there is a better tool for some of those contexts.

And here in lies the tragedy of the situation. Organisations attempting an Agile Transformation are seeking certainty. Their value model does not allow they to value a coach using option thinking, i.e. how many knowledge options they have to  acquire skills to solve a problem. Instead they value coaches based on their expertise and their popularity. Until Organisations learn to value coaches properly, they will continue to suffer failures in their transformations.


Agile – The Broken Learning Machine

In my last post, I introduced two types of community. Communities of Need centered around solving problems, and a Communities of Solutions centered around selling solutions. The nature of these communities drive different types of behaviour. Neither is good, and neither is bad. They are just different. Whether the behaviour is appropriate is all about context.

The Agile Community started out as a learning community. Members of the community were highly linked, able to call upon other members when they became stuck or wanted some expert opinion. I was lucky enough to attend Agile Development Conferences in 2003 and 2004. For me, the most exciting sessions were those in the Open Space. You could discuss your tropes (apparently meme is a bad word) and get the most amazing feedback. I remember being disappointed that my first session on Real Options only attracted about eight people ( how was I meant to know who Rebecca Wirfs Brock, Steve Freeman, Eric Evans were? ), however I received amazing feedback (funny that). At ADC, the whole conference closed down for Open Space. This forced everyone to attend, or go for a walk. I still follow the ideas raised in the project management session that came up with the idea of a behavioural coach. Open Space broke down the barriers so that people who were not experts in Agile could talk about subjects that they were experts in.

Credit has to go to Alistair Cockburn and Todd Little for creating an amazing conference in ADC. The goal was to have “Three days to start a conversation”. Building community was at the heart of the conference. To this end, they created the ice breaker as a means to break down barriers and get people to introduce each other. Agile2009 was the last conference I attended by choice ( Kent forced me to go in Agile2013 ). At Agile2009, the ice breaker was little more than a way of enticing attendees into the sponsor’s lair with the lure of a few free drinks. The big conferences need to be about building community as well as selling selling selling ideas.

Many of the Agile practices map to Kolb’s Circle of Learning. In particular, the retrospective that is at the heart of improvement for many Agile teams.

Like others I skipped Agile2005 but unlike others I was drawn back in for Agile2006 in Minneapolis. Open Space was relegated to a corner of the conference next to the canteen. Hidden behind the growing ghetto of sponsor’s booths. Already the problems I discuss below were evident. Bil Kleb and I co-hosted an open space session on “Learning, Cognition, and the Scientific Method”. I explained how the Agile Community could be mapped to Kolb’s Circle of Learning, and the behaviours that were detracting from the learning. Here is how community learning maps to Kolb:

  1. Spot the Example: “Yo dude that was awesome, let me try!”
  2. Model: “Lean forward yo!”
  3. Test: “Cool dude!”, “Major face plant loser! You did it wrong!”
  4. Reflect: “This work most of the time! Lets do it”

In the next cycle.

  1. Spot the Example: “Lots of people are face planting the same way”
  2. Model: “Lean forward except in powder, then lean back.”
  3. Test: “I hope he face plants! Face plant! No!!!! He pulled it off, even in powder.”
  4. Reflect: “Much better. Only a few face plants now. But they really hurt so lets look out for them.”

You can travel around Kolb’s loop forwards (as above) or backwards (Feature Injection’s “Break the Model”)

Even in 2006, the following behaviours were becoming evident that detract from the learning.

  1. People were not joining the Agile community. They were reading books and turning up to just listen. They were not engaging.
  2. Conferences were seen as a corporate jolly rather than a community “Gathering of the Tribes*”. Attendees stuck together rather than meet new people.
  3. Consultants were keen to prevent their clients from meeting other rival consultants.
  4. We had no way of capturing the fails. Those attempts that resulted in failure.
  5. Gladwellism was rife.
  6. People are doing it wrong became a trope.
  7. Some thought leaders started to get angry at the mention of community. These are solutions to sell.

Agile was rapidly shifting from a “Community of Needs” to a “Community of Solutions”. We started to hear talk about “Crossing the Chasm” as the goal. This goal assumed you wanted to sell Agile, rather than you had a need to solve problems.

The worst problem was number 4.

Capturing the Fails

Between finding a solution and reaching the chasm, the uncertainty surrounding a Trope has to be resolved. This means that those working with the idea have to be on the look out for failure masked as success, and for failures that are occurring silently. Community of Solutions need this to cross the chasm, and members of the “Community of Need” want the failure stories to improve the idea so that it fits more contexts.

Consider the following scenario. I give a presentation on Real Options. One hundred people decide to try out real options. They have the following success:

  • One group have huge success and come back to the next presentation where they tell everyone how great real options is.
  • One group have moderate success. Enough to keep using it, but not enough to go to another presentation or tell their friends.
  • One group has so so results. They use it when its obviously going to work but do not talk about it.
  • One group has moderate failure. They tell their friends and family to avoid.
  • One group has huge failures. The failures are hushed up, or converted into success stories that create a new generation of evangelists. This leads to even bigger failures later on.

Which group do we hear from? The “huge success” group of course. But why did the other groups fail? What was it about their context that caused real options to fail? We need to know this for two reasons. Firstly, we need to warn people when it will fail so that they can use it safely? Secondly, we need the failure stories so that we can improve real options for those contexts where it fails, and ultimately cover more contexts? ( An aside: At one conference someone introduced me to an interesting idea. I asked when it failed. “Never. It had not failed them in a decade”. Suffice to say I never tried it. Too risky if the failure states are not known. )

When we do encounter stories of failure, it is normally at a “Non-Agile” event. “We tried that”, says someone at an IIBA or PMI or XYZ event, “and it did not work”. We then work with them to analyse the situation so that we can show that they failed to apply real options correctly, rather than real options failed. In truth, If they applied real options incorrectly, it is a failure of real options most of the time. A failure to warn people what the failure states are.

The pressures in a Community of Solutions result in behaviours that work against the learning in a Community of Needs. After all, why would an expert in a subject admit that they are not the real expert, and the attendee on their course should speak to someone who does it, rather than someone who helps others do it.

So the Agile Community as a Learning Machine is broken. We need an effective feedback mechanism to capture failure stories. To harvest those stories where the context ate the trope.

This is the second post building up the content for my ALE2015 keynote.


Classifying Insight Assessment

As part of the work that Chris and I performed on the product management framework (ref the link), we tested it for fit against a series of different sources of insight. Interestingly, we found that the three branches of Business Outcomes, User Needs and Target Groups were sufficient for progressing to design concepts; the trick was identifying the order in which the disciplines are involved.

The table we came up with is shown below. While it might seem curious that, for example, an iPad starts with the business outcomes, we began here because it’s clear that Apple would have a very clear view of what they wish to achieve from a future product and would build their product to those KPIs. They have a clearly identified target segment for new products and subsequently focus on meeting that group’s needs. Note that leaving the User Needs until last does not imply that these are subservient to the remaining disciplines; rather that the met needs will be focussed toward driving the KPIs for the identified segment.

Types of Insight

 

We do not show Technical Insight here. This was a recurring theme in the recent book “How Google Works”. However, a technical insight makes something possible that was previously not possible (or cost effective / adaptable). One must still meet the needs of a target group for a given outcome, even if the ultimate use is not apparent at the start of the process. Thus technical insights help us prioritise; however, we still create products that meet the needs of a segment of users, in line with our business goals.

Note that business analysis is the correct approach for Technical Improvements or Regulatory Compliance. These are the role of domain experts – the Complicated space in the Cynefin model – rather than the safe-to-fail experimentation of product management as befits the Complex space.

We’d love to know whether this tallies with your understanding of how insights can be used. And are there any missing cases?


Communities of Need & Community of Solutions

Much of my thinking recently has been on Culture in Organisations. I am also interested in the Culture in Communities, and in particular Communities of Practice like Agile and Kanban. My thinking has lead me to observe that there are two distinct communities within a Community of Practice. One community based on solving problems, or a “Community of Need” and the other based on promoting solutions or a “Community of Solutions”.

Context

Crossing the Chasm by Geoffrey Moore is a well established way of looking at technology adoption. It is so well established that Seth Godin rebranded it as Purple Cow for the Marketing Community over a decade ago. To summarise, through out its life time, a technology will be adopted by “early adopter”, then the “early majority”, “late majority” and finally the “Lagards”

crossing chasm

Between the “Early Adopters” and the “Early Majority” is the Eponymous Chasm that many great ideas have fallen into, never to be seen again. “Crossing the Chasm” is about getting an idea across that Chasm to the riches and success on the other side.

It has helped my understanding of this to overlay Julian Everett’s excellent work on “The Meme Lifecycle“.

Meme Lifecycle

From Julian’s Meme Lifecycle, all meme come about as the result of a need. Before the meme or solution exists, there is simply a need*. There are often several independent groups or individuals working on solutions to the need which it is why there are often several independent and competing solutions that occur at the same time. Once a meme has come into existance, it goes though a period of evolution and natural selection. During this phase, the meme is tested in different contexts, or fitness landscapes. In each fitness landscape, the meme might take a different form or expression known as an extended phenotype. The key issue about this stage is that a great idea might fail because of the context, and as a result it will need to evolve to satisfy the needs of that context. As a meme approaches the chasm, the uncertainty about the contexts it will and will not work in is resolved. A necessary condition of a meme exiting the chasm is that there is certainty not only about the meme and how to use it, but also which contexts it will and will not work. People who claim that a meme fits all contexts and then modify it to fit a new context are preventing the meme from exiting the chasm. In my opinion, Scrum needed Kanban to leave the Chasm.

For example, Recently someone asked Richard Warner and I about Scrum and Kanban. In fact, they knew what they were and how to do them, what they wanted to know, was “when to use Scrum or Kanban”. Richard came out with the following conditions:

  1. In order to put work into Scrum, you have to be able to estimate it (or time-box it). This means for discovery (PM/UX) work, Kanban is a preferred mechanism.
  2. Scrum is only possible when the work is predictable for at least the next week or two. For highly unpredictable work, such as IT Operations or UAT, Kanban is preferred.
  3. Kanban works best when the person responsible for prioritisation (Product Owner) is highly available, preferably co-located. In situations where the prioritiser is not highly available, it preferable to batch requests using Scrum. So for an off-shore development team, the PO team might use Scrum to manage prioritisation. The development team might use Kanban inside the Scrum framework.

Its is now known when to use both Scrum and Kanban most of the time. This makes them far more appealing to the risk averse who hate uncertainty.

Next, I overlaid Dave Snowden’s Cynefin. Doing this helped me link to the cultures associated with each part of the curve. (More to come later on this)

memes cynefin

Cynefin maps nicely to the Meme Life cycle. From creation of the meme to exiting the chasm, agents engage in safe to fail experimentation to test multiple hypothesis in order to map the meme to the different fitness landscapes.

Mapping to Cynefin made it easy to map to Real Options. An option’s value is only greater than the intrinsic value when there is uncertainty. When there is certainty, there is no value in deferring a commitment. This is where “unless you know why” comes in. If you know why, if you have certainty, then you can commit early.

cynefin real options

And finally, it is possible to map to principles versus practice.

real options principles

Those operating before the chasm act based on principles. They understand that they may be operating in a fitness landscape where the meme they are using fails. Once the meme has exited the chasm, the users expect that they only need to follow the practices, and the principles become less important. Often the principles are forgotten or ignored and this leads to failure as more obscure fitness landscapes are discovered. This failure corresponds to the cliff in the Cynefin Model. Fundamentally, principles should dominate before the Chasm, and practice dominates after the chasm. This causes real problems for memes that are principle driven like many of the Agile practices.

Communities of Practice

There are two clear communities of practice that overlap considerably:

  • The “Communities of Need” are centred on the creation of memes. Members of this community operate in the area of “Need” to create a meme and then work with the meme to identify fitness landscapes where it fails, and evolve them accordingly. These communities have problems that need to be solved. They take solutions developed for one context and attempt to implement them in their own context (exaption), and modify (evolve) them as appropriate.
  • The “Communities of Solutions” are centred on the Chasm. Members of this community attempt to extend the number of fitness landscapes that a meme works in and then sell the meme to the early and late majority on the other side of the chasm.

Communities of Need

Communities of Need can easily be identified by their gatherings. For example, conferences would consist of the following:

  • An un-famous Keynote, often talking about ideas that are unfamiliar or novel to the community. The keynote has practical experience of the material they discuss. The keynote is not normally a draw for the conference., (The gathering of the community, and exchange of ideas is often the draw.)
  • Experience Reports dominate, and spark discussion. There is an understanding that Communities of Need grow based on experiences. An experience prompts others to share stories and a “model” is built accordingly.
  • The conferences have a heavy open space element as the participants are often as knowledgeable as the speakers/facilitators. Organised sessions are often workshops ( Gold fish bowls ) to harvest community knowledge.
  • Context is everything.

Examples of “Community of Need” conferences include XP Day, Lean Agile Scotland, Agile Lean Europe 20xx, Lean UX NYC.

Once a need has been resolved, communities of need may dwindle, or move on to solve another need. A classic example of this is XTC. The needs of the London XTC community have pretty much been met. They know how to write software in small teams, and there are no significant community needs outstanding. As such, those interested in personal development now gravitate towards the software craftsmanship community.

Every member of a “Community of Need” is a leader. They are Problem Champions. As such, these communities do not tend to have leaders. If anything the leaders are often servants of the community who support the community, organising events but no one knows who they are.

Communities of Solutions

Communities of Solutions are about promoting meme across the chasm and beyond. Their conferences are also obvious:

  • The keynote is famous. More often than not talking about a subject they have no practical experience in. They have either read a book or done a bit of study. Malcolm Gladwell is the ultimate expression of a keynote for a CoS. If they do have any experience, it dates back years in the past.
  • There are normally workshops added onto the conference on subjects near the chasm.
  • The sessions are organised in advance and consists of talks, workshops and “taster” sessions where the attendees learn from the speakers. Attendees want to be told what to do.
  • The Conferences are organised as commercial ventures with the profits generated going to the organisers.
  • Context is not considered relevant.
  • “Community of Needs” individuals are normally the speakers and hold small impromptu conferences in “The Corridor”

Examples of “Community of Solution” Conferences include Agile20xx, QCon, Goto.

These communities have leaders with a clear pecking order. The leader often annoints the next generation. The leaders of a “Community of Solutions” are often not practitioners or even members of the “Community of Need”. Their position is normally due to their relationship with the existing leaders. “Communities of Solution” have leaders and followers. The leaders are the Solution Champions.

Scaling Agile

So where am I going with this?

The Communities of Need in the Agile Community are in a terrible state. Very few members of the Community of Need attend Agile 20xx anymore. There just aren’t enough practitioners around to create a decent corridor conference.

There is a belief that Agile is crossing the chasm, or that it has crossed the Chasm. Merging these different views has helped me to understand the following:

  1. Some Agile Practices have crossed the Chasm.
  2. Agile has not crossed the Chasm.
  3. Scaled Agile certainly has not crossed the Chasm.

Picking on Safe as an example of Scaled Agile, there are three parts:

  • Team level Agile – This has crossed the Chasm. Safe provides a nice packing of the established Agile Practices.
  • The Agile Release – This is halfway toward the chasm. There are several successful case studies.
  • Portfolio Vision – Agile at this level has not even identified all the needs. There are still several problems to be identified.

So some of the Safe practices are across the chasm with the uncertainty resolved. Others have not. You need to hire people from the “Community of Solutions” to train and coach your organisation. That is not enough though. Any company considering an Agile Adoption at Scale should also hire a number of people from a “Community of Need”. The people from the “Community of Need” will help you with the context issues. They will also help you create safe to fail experiments for those problems where there isn’t a known solution. To misquote Dave Snowden “You need Chefs as well as cookery books”.

I’ll end there. This is part of the Keynote I’m preparing for Agile Lean Europe 2015. My next post will be to discuss the problems caused by the lack of structure linking these two sets of communities.

* This is known as the entrpreneurs in “Crossing the Chasm”


Risk Adjusted ROI of Agile & Waterfall – A simple example

Inspired by Ron’s Comments on the Risk Adjust ROI of Agile and Waterfall post, I wrote a simple example to illustrate the point I was trying to make.

The expected ROI of an investment is often put into a spreadsheet where the present value of future expected cash flows is calculated using a discount factor based on the appropriate interest rate.

The ROI for a Waterfall Project:
equals
Discounted returns of expected future cash flows.
minus
Project Costs

The ROI for an Agile Project:
equals
Discounted returns of expected future cash flows (Higher due to getting cash flows earlier)
minus
Project Costs (Higher due to additional testing & release costs)

The higher value of discounted returns due to getting cash flows earlier is not that significant, especially in low interest rate conditions such as we are currently experiencing. The expected project costs are much higher as the testing and release will happen more times. The person putting the spreadsheet together would rightly assume that if we can reduce these costs, we would do so in both scenarios.

As such, Agile does not fair so well when compared to Waterfall using expected ROI.

Now lets consider the Risk Adjusted ROI.

The RAROI for a Waterfall Project:
equals
Discounted returns of expected future cash flows (The cash flows should be risk adjusted as well as simply discounted due to interest rates, resulting in a much smaller value).
minus
Project Costs

The RAROI for an Agile Project:
equals
Discounted returns of expected future cash flows (Higher due to getting cash flows earlier, especially given risk adjustment)
plus Option to increase investment
plus Option to change requirements
plus Value realised by reducing uncertainty through the tests
minus
Project Costs (Higher due to additional testing & release costs)
less Option to terminate early without losing investment to date
less Option to release without having to manually test

Using RAROI results in a much higher value for the Discounted Cash Flows in Agile than Waterfall. This is because the risk adjustment would be more significant and would have a great impact on the Waterfall cash flows that occur at the end of the project than the Agile project where the cash flows occur earlier. The risk adjustment would take into account for things such as:

  • The longer time before the product reaches the market increases the chance a competitor releases a similar product.
  • The longer the project is running, the more chance it has of being cancelled.

The options cannot be valued accurately. Only snake oil salesmen and fools would advise using Black Scholes. Instead, we make up some values for illustration purposes only. A simple option payoff multiplied by the probability will suffice.

Option to increase investment

If the value of the investment is shown to be significantly more than expected, more can be invested.

e.g. 10% chance of Value being double that expected.

5% chance of Value being ten times that expected.

Option to terminate early without losing investment to date

This option would save an amount of the costs. This option requires that the people on the project can be re-deployed onto something useful instead. For simplicities sake, lets assume that this relates the forty percent of feature never, or rarely used according to the Chaos Report. i.e. Forty percent of costs of the project.

Option to change requirements

So this relates to those forty percent of features cancelled. That money could be re-invested in valuable features instead. A potential up-tick of forty percent in the value.

Option to release without having to manually test

This is a particularly valuable option as it improves the resiliency of the system. Emergency production fixes can be put live without the risk of putting untested fixes into production, and without the delay of manual tests.

It can be seen that the Options in the RAROI create a significantly more robust argument for using Agile than Waterfall.

In summary, RAROI shows that Agile is superior than Waterfall whereas ROI does not. Often people think they are using ROI when in actual fact, they are using RAROI. Whilst it does not make a difference to normal people, it does to people who are experts in finance.

Thank you Ron for your thought provoking comments.


Risk Adjusted ROI of Agile versus Waterfall

“In theory there is no difference between practice and theory. In practice there is.” This week I saw one of those examples.

ROI of Agile

Nice.

ROI $$ (agile) > ROI $ (waterfall)

A great compelling argument for those promoting agile over waterfall. The only problem is that it is naive and WRONG. The argument can only be had in retrospect. The organisation runs two projects, one agile and one waterfall, and compares the ROI on the two projects. I am not aware that anyone has run such an experiment. It is a religious belief without proof.

In reality, the comparison of the two approaches is done before the project starts. The comparison is made on the expectation of the ROI. Those performing the evaluation compare what they expect the ROI to be. The fact that they are considering waterfall indicates the project has a knowable outcome and is in the complicated domain of the Cynefin framework. The functionality is known. If at this point, it is known that the functionality could be reduced, then it would for both. However, when we look at the costs, waterfall only has one test and production release and agile has lots. Given the information at the time, Agile will cost more compared to the waterfall approach. The fact that they are considering waterfall also would tend to indicate a risk averse culture at play which means they wont consider risks or changes.

In summary

Expectation ( ROI (agile) ) < Expectation ( ROI (waterfall) )

Which means that if you encounter someone competent from Finance or a Finance background and start talking about ROI, you are going to sound like an idiot. Someone will use your lack of understanding to undermine your argument to their advantage.

Now lets consider Risk Adjusted ROI. Risk Adjusted ROI is exactly as it sounds. It adjusts the ROI for risk. There are many types of risk associated with an IT investment. Steve Freeman and I wrote an article in The Agile Times (published by the Agile Alliance in 2005) where we list the three categories of risk.

  1. Delivery risk – Failure to deliver the investment within the agreed constraints
  2. Business Case risk – Failure to deliver the identified value
  3. Damage to the Existing Business Model – The investment damages the exisitng business model

Agile can be used to manage all three categories of risk better, but only if it is applied properly. A Scrum or Kanban project that only releases once a year is not managing risk (and is not really a Scrum or Kanban project).

Now lets compare the expectation of an agile and a waterfall project. All of a sudden, all of the real options that exist in an agile project and make it less risky, can be considered in the valuation. i.e.

Expectation ( Risk Adjusted ROI (agile) ) > Expectation ( Risk Adjusted ROI (waterfall) )

And this is where theory and practice come in. In theory, the ROI argument is good enough. You read a few books and write down that ROI is the way to compare agile and waterfall. The practitioner tries this out and it does not work. They then have to dig deeper into the theory to come up with a something that does work. “As simple as possible but not simpler”. “A simple as possible” is the battle cry of the theorist. “but not simpler” is the retort of the practitioner. Finding “As simple as possible but not simpler” for your context can only be determined by praxis, the co-evolution of theory and practice.

For me, a humbling example of this on a personal level was when I was trying to come up with a definition of business value in 2003. I read “The Goal” by Eliyahu Goldratt and thought “A project creates value when it increases revenue or reduces costs”. I asked the business manager on the trading floor and he said. “Nope! Eighty percent of our investments are to protect revenue, either retain customers or build risk management systems.” – To quote Jabe… “Praxis makes perfect.”


Follow

Get every new post delivered to your Inbox.

Join 78 other followers