Value and Metrics

A significant risk in any business is that you do not know where you are. Value is the thing that we aim to deliver, but value data or metrics are how we measure our success at achieving our goal. The only way we know for certain that we have achieved our goal is when the value data shows us success. For example, we aim to deliver $2million from a feature. The only way we know we have achieved that is when we have the $2million in our account.

Simple right?

Nope.

• How do we know that it is the new feature that has delivered the value?
• Do we have to wait until the money rolls in until we know how we are doing?

If we have a single feature that people are buying it may be fairly simple, but that is rarely the case as most applications have many features. We have to put more thought into how we look at our value data. First, we need value data on how the user is using the new feature.

• Do the users use the feature? Did the
• How often to do they use the feature?
• For how long do they use the feature?

Our value data impacts how we might introduce a feature to our users. Do we offer a couple of free goes before they have to pay or do they pay before they use? Offering a couple of free goes allows us to gather value data (or metrics) on how people use the feature. We want to know how long the user spends on each step of using the feature and whether they stop using the feature before they complete all the steps. This will help us understand whether the user values the output enough to put the effort in to provide all the inputs. The value data will tell us which step result in people dropping out of the process.

Context is everything. Context drives how you use value data. A web site is different to an installed application. Retail is different to enterprise. The value of each customer impacts how much we are prepared to test on them. The golden rule is to get feedback on our business case.

If we have an application that the user can download for free and have a couple of free goes before they pay, we have to build a business model ( or belief system ) around how we will make money.

For example, we assume 1,000,000 download the application. 50% use the first free go with the feature, of which 20% use the second free go, of which 10% then pay for the feature. The feature costs $10.00. This would result in revenue of 1,000,000 * 50% * 20% * 10% * $10.00 = $100,000. We could release an earlier version of the feature which tests to see how many people download and use the free versions. We can look at when people drop out of using the feature.

We aim to deliver value but the value data (or metrics) let us know whether things are going to plan.

Value is not enough, we need to consider the return on investment.

Return on Investment breaks Feature Injection.

For many years I have focused on value as the driver of IT development. “Break the model” is the part of Feature Injection that tells us to look for examples that do not fit within the “Olaf” (Model). For some time I’ve been aware that there are examples of development investment that do not fit nicely within the Feature Injection process. These examples have been at the edge of my peripheral vision. I’ve been aware of them but almost subconsciously ignored them because they do not fit in the model. This is exactly the bias that “Break the Model” attempts to address.

What are they? Improvements to the inputs and processes (expecially non functional requirements). Obvious they are valuable but they do not deliver value to the user as we know all of the value in a system comes from consuming the output. i.e. The outcome.

According to the definition of Feature Injection, all value comes from the output which is true, however improving inputs and processes also brings benefit to the user which is also true.

I have been considering a number of successful companies which do not have an obvious source of income. For example, twitter and instagram. For these companies, they value increasing the number of people who have installed their software (network size) and the amount that people use their network (network activity).

Users of an application will probably download it because it is valuable to them.

The amount of usage will depend on the return on investment of the application. This is a function of the value of the application (which is in it’s output) and the investment they need to make in order to get that value.

The investment from the users perspective is determined by the effectiveness of the inputs and processes. From the users perspective, the investment takes many forms, some of which are:

• Money ( to buy the software / service )
• Time ( How long you must work to get the value you seek )
• Mental investment ( How much brain power is required to derive the value)
• Delay ( How long you must wait for the value )
• Frustration ( How easy and forgiving the application is to use. Whether the app play “snakes and ladders” with the user. )
• Training ( How much time and effort must be invested learning to operate the application )
• Transferability ( Whether the investment is transferrable to another context or application )
• Specialisation.

This leads to an extension of the rules for generating business value.

Development generates business value when it reduces the investment that a user needs to make in order for them to acheive the value they desire.
This is an addition to the existing definition rather than a rewrite. It also means that Feature Injection will need an additional step so that it becomes.

1. Hunt the value (i.e. Increase value)
2. Inject the features
3. Break the model
4. Reduce the user’s investment.

Example – Cameras*.

Cameras have no independent value in the process of image making. The value of a camera is to capture images that can be shared with others across time and location. The value is in the images that are viewed at some later date.

The value in the process has never changed. The value is in the viewing of the images. If we cannot view the image, then the camera could easily be replaced with a stone with loss of value to value stream.

In the early days of cameras a low quality photo image would be created by a very expensive process that involved learning optics, chemistry and building a dark room. This meant that the process was limited to a very small number of people who only captured very high value images (The Queen’s Wedding and Coronation).

The photo production process was automated but still specialist. This meant that people could use a camera to capture images but there was a delay of days to recover the images. The cost of investment was reduced so we could capture images of lower value (Normal people’s wedding’s, birthdays and holidays).

One of the problem with the process was a lack of forgiveness. You never knew whether your images had been captured. This changed with the introduction of the Polaroid Camera which provided instant feedback. You could now take photos until you got it right.

The electronic camera wiped out much of the production process and thus cost of the images.  (Now every day events could be captured).

The internet and social media sites wiped out the remaining cost of getting the images in front of your intended viewers. (Even trivial events can be shared at very little cost).

This has lead to an exponential increase in the number of captured images.

Feature Injection was broken. It failed to properly incorporate non functional requirements related to inputs and processes in the investment decision process. We can fix this now and start to look for more examples that do not fit.

*Deliberately chosen to get a response from @Cyetain

Its not a model, its an Olaf.

At ScanAgile in Helsinki (great conference), Olaf Lewitz pointed out that the “model” in the “break the model” part of Feature Injection is not a model as we would normally describe it.

My understanding of the term model is “A simplification of reality”. This was my understanding when I started using the term.

Olaf pointed out that what we had in “Break the model” was “A summary of examples”. The important part is that the process used to create it is to incrementally add examples and adjust it as we add examples that are different in some way. We do not create our model by simplifying the reality of the situation. We do not have a good word for this and so we are calling it an Olaf until a better name is discovered. In effect, we now have “Break the Olaf” in  Feature Injection which seems a bit harsh on Olaf, so I think we will stick with “Break the Model” until we find the right name.

So what is an Olaf? An Olaf is a set of examples that represent our reality. We use it as a filter to identify new examples that do not fit in the set.

e.g. We have a white square and a black square in our set. We then spot a red square. We know that this does not exist in our current set so we add it.

In its simplest form, the Olaf is a the set of examples itself. However, it becomes hard to spot new examples by comparing them to the existing set of examples. To make it easier to spot new examples, we abstract properties and behaviours.

“Break the model” means we have found an example that does not fit into our set of examples.

e.g. We start with a black square.

Our Olaf is a black square.

We spot a white square which breaks the Olaf.

Our Olaf is now a black or white square.

We spot a red square which breaks the Olaf. We abstract black, white and red into “coloured”.

Our Olaf is now a “coloured” square.

We spot a triangle which breaks the Olaf. We abstract triangle and square into straight sided shapes.

Our Olaf is now a coloured “straight sided shape.

We spot a circle……….

The key thing is that the Olaf describes our existing examples so that we can more easily spot new examples. Once we spot an example, we simply add it to the list of examples. To make things easier, we may create an abstraction of the examples.

Let me know if you think of a better name for an Olaf.

QCon New York

Just finished the initial draft of slides for my presentation at QCon New York. Will be tweaking, adding and pruning over the next few weeks as I remove content and hope to add humour.

Amr Elssamadisy asked me to put together a track on the “Agile Individual”. I did what any sane person would do and invited the people I want to hear speak. The only constraint was they had to talk about Agile Individuals.

Mike Hill is a legend from the early days of XP. He will be talking about how individuals make amazing teams. If he does not offend, then he will have to try harder. Mike Roberts is just super super smart. He will be bustin’ and bruisin’ some Agile processes in the hope that the focus will be on individuals instead. Matt Wynne did a hilarious talk at Cukeup last year. The additional constraint I placed on him was that his talk had to be just as funny. If not, I’ll banish him to a cave in Scotland. I’ll be talking about Learning… and how to do it the lazy way. Sue McKinney will be on Late ( a private joke ) talking about the Executive view of the individual in Agile.

All will be excellent presentations except mine which currently has four hours of material that I hope to squeeze into 45 minutes.

Risk – Its all about time!

I was talking to a colleague about risk. They mentioned that they list the risks and assign a probability and impact  to each risk. This appears to be the standard approach taught on many management courses.

“I take a different approach” I said “I only consider impact”.

I am interested in two measures.

• How long it will take to fix the problem.
• How long the business can survive with the problem.

If the time to fix the problem is less than the time the business can survive, we do not have a problem.

If the time to fix is greater than the time available, then we need some more options.

Olav and I mention a number of examples in our InfoQ risk article

Can you think of other examples?

Is TDD always the best solution? The third way?

Recently I have been involved in an “Agile” transformation. I have been forced to look at a number of teams and assess which tools from the Agile toolkit are relevant.

Rather than simply foist Agile goodness on the teams, I’ve been engaged in a discussion about the relevance of the tools… especially from a risk perspective (what a surprise!).

To help people understand Agile I have been casting it in a historical perspective against the (perverted version of) Royce’s Waterfall approach with sign-offs and change control mechanisms. Adopting an iterative development approach mitigates the need for those sign-offs and change control. In addition, the iterations help business investors mitigate a whole bunch of other risks.

Multiple iterations cause some additional issues. There are transaction costs associated with an iteration such as regression testing and releases. Without careful management, some of these transaction costs (regression testing) will increase with each iteration which will start to have a negative impact on the teams ability to deliver new functionality.

The traditional approach is to perform extensive manual regression testing. This increases with each iteration and as a result the economics of the situation lead to larger iterations which breaks the risk management goodness of small iterations.

The agile approach to this problem is to automate testing. To ensure comprehensive and effective coverage, it is necessary to consider testing as a primary concern. Thus test driven development comes to the fore.

The team I work on has an interesting context. The system is not seen outisde of the organisation. It is not used to make payments to external entities. The users do not use the information solely to make decisions. As a result the users are tolerant to bugs… provided they can be fixed quickly. The team can turn around bug fixes within a few minutes to an hour which is acceptable to the business. So this presents a “third way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly (including by roll back which has not yet happened).

It will cost the business investor a lot of time and money to retro-fit effective automated tests. It will take time and effort for the team to develop the skills to learn TDD and build automated tests around the legacy code base. Time and effort that the business investor could spend on other things.

Currently the team are able to produce several releases a week. The business investors are happy with the performance of the team.

Would TDD be the best approach or would the “Third Way” be more appropriate to this context? Would the introduction of TDD break the existing development process from the business investor’s perspective?

Should TDD always be foisted on a team? Or not? Would a team be Agile if they did not do TDD?

Thoughts?

P.S. I am aware that TDD brings many other benefits beyond regression testing.

Forgiveness

I met my first User Experience designer in the late nineties. My boss hired a guy with a PhD in the subject. I asked him “What is the most important thing in interface design?”. His answer “Forgiveness”. He explained that Forgiveness was a UX concept that allowed the user to gracefully recover from failure. Without me knowing it, he was one of my earliest mentors in the field of real options.

I now consider Forgiveness to be an indication of a high quality system. When your interaction with the system fails, does it do so gracefully, or does it require you to jump through a number of painful hoops until you get back to where you were originally? Of course, the only way to see how a system handles failure is to experience failure. Until you experience failure, you do not know whether the interface will perform as described. Many systems claim to handle failure gracefully but not all system do. If you really want a deep understandinf of a system, you need to see how it fails.

When I think of systems, I think of People and Relationships and Groups and Organisations and Communites and Societies, oh and Computer Systems. For me, the most interesting systems are those that contain people ( Complex Adaptive Systems ).

Sometimes, failure in one of these systems means expulsion or a loss of status. These systems discourage people from experimenting, or playing to close to the edge of chaos. By doing so, these systems discourage deep learning at the edge. They discourage people from seeking out and challenging the axiom ( or beliefs ) of the system. The Catholic Church’s original treatment of Gallileo is a classic example of this. Eventually the Church forgave Gallileo but it might have been five hundred years to late for him to care.

Just over two weeks ago I attended the CALMalpha event. This event hoped to bring together Cynefin, Agile and Lean communities to explore whether there were any ways that they could help each other learn. For me it was a bust. I did not really learn anything* as I was already familiar with most of the material. I had fun and I enjoyed meeting old friends and new. Meeting new friends and old justifies the attendance. Although I did not learn much I am grateful to the faculty for making the event happen. In particular Joseph Pelrine who prepared a lot of new material to describe his journey with Cynfein.

Several tweetable soundbites (you can check them on twitter I think ) spring to mind from the event about the Cynefin version of social complexity…

• Seek out the cynics as they really care. The rest are just corporate survivors.
• Experiment in a “safe to fail” environment.
• The only way to understand a complex environment is to act within it.
• Ritual Dissent uses ritual to protect people from negative feedback.
• Complex situations are understood in retrospect.

As I journeyed home I compared Ritual Dissent to Feature Injection’s “Break the Model”. One advantage of Ritual Dissent is that it provides a mechnism for those with negative intent to feedback on ideas whereas otherwise they may not. I had not realised how important the ritual was to the process.

A significant proportion of the activity in the Agile and Lean Communities takes place on line in e:mail groups, linkedin groups, blogs, Skype and twitter. Conferences are an opportunity to re-energise and build new collaborations. Conferences are not the conversation, they are a high bandwidth blip in the conversation.

The Cynefin community seems to be much more close knit and more based on face to face contact with the central leadership. Joseph’s social network analysis seemed to indicate as much. This is possibly because most of the intereaction is through courses to date rather than through conferences and meetups.

Here are my retrospective thoughts.

Prompted by Willem’s blog, I wrote up my experiences of the CALMalpha event. I did it in the style of Ritual Dissent. In effect, I acted within the system to understand it.

I failed. And I apologised. I failed to apply Ritual Dissent correctly. Did I do it intentially? No. Did I pay attention properly when it was explained? No, I was hungover and too busy avoiding having to take my turn.

I was the cynic. I was the one to be sought out because I cared. If I did not care, it would have been much easier to keep my comments private.

There are a number of questions in my mind.

• Did I fail in a safe environment?
• Does the Cynefin community really value cynics?

and of course…

• Does the Cynefin community exhibit forgiveness?

Last week I went to #ScanAgile where Dave Snowden and Jospeh Pelrine were also speakers. I was more than a little uncertain how  things would play out as my failure had been pretty spectacular. I had a wonderful chat with Joseph Perine about the whole episode at breakfast. I was then fortunate enough to have a walk around Helsinki with Dave Snowden, Mika Latokartano (who acted as our guide) and Olaf Lewitz. Dave treat Mika and I to a wonderful luinch before Mika drove Dave and I to the airport.

Does the Cynefin community exhibit forgiveness? That’ll be a bug yes.

I learned a lot since the Cynefin event. I look forward to helping the CALMbeta event (from a distance as I limit my travel to the USA).

* not quite true. I did learn a lot from Jabe Bloom about photography. His ideas have already started to affect the way I take pictures.

Feature Injection, The Meme Lifecycle and Serendipity

For me, Serendipity is a beautiful happenstance. A series of events that lead to a beautiful and astounding coincidence. Yesterday I stood in the Carl Larsson exhibition in Helsinki and shed a tear at the beauty of what I beheld.

“Break the Model” in Feature Injection is a process for learning. It is effectively David A. Kolb’s experential learning circle. “Break the Model” has four steps. First, spot an example. Second, reflect on whether the example is relevant and/or if it is actually a new example. Third, create a test by detailing out an example (and store the example). Fourth, create a “model” which is used to help you identify other examples (step one) that will break your model. On Thursday Olaf Lewitz helped me present on Feature Injection at ScanAgile. Olaf had realised that we are not creating a model which is a simplification of reality. Instead we are creating an “Olaf”** which is a summary of examples.

As well as using an “Olaf” to spot new examples, it can also be used to spot more examples of the same kind. A few years ago I created just such an “Olaf”. Julian Everett’s Meme Lifecycle helped me understand certain forces at work in the creative world. The example I gave in the comic strip is “Flowers on Windows”, a meme that existed in the work of Charles Rennie Mactintosh and Frank Lloyd Wright. In actual fact, the meme should have been called “Flowers on Light”. Yesterday I saw the meme in the fabrics designed by Carl Larsson’s wife Karin Larsson. I immediately saw the similarity with the works of Charles Rennie Mactintosh and Margaret Macdonald (Mackintosh’s wife).

Why is the meme significant?

Agile is based on the patterns group which evolved from the work of Christopher Alexander. At Agile2009, Kent MacDonald and I visited the home of Frank Lloyd Wright in Oak Park. It was like stepping into Christopher Alexander’s Pattern’s Book. The similarities to Mackintosh’s work were obvious, especially the “Flower on Windows” meme. If Alexander was Agile’s granfather, Wright was it’s great grandfather. But where did Wright get his inspiration? I now had a meme I could use to trace the influences.

If I could spot it, I might find a another clue to the orgins of Agile. Yesterday I spotted the meme in the textiles of Carl Larsson’s wife Karin. A massive smile appeared on my face. I walked around the corner of the exhibition and was stunned by what I saw? A board covered in post it notes. Now that’s what I call Serendipity.

In the next room of the exhibition, the organisers had blown up Carl Larsson’s pictures to create a massive Wendy House maze where children were dressing up and creating pictures.

“Flowers in Light” is a meme that responded to the inhumanity of the industrial revolution. It was a way to bring nature (flowers) and light into the cities. The parallel’s to agile are obvious. Carl Larsson is a new example. I now need to reflect and research to discover whether he was an influence on Wright and Mackintosh or whether he was simply responding to the “Flowers in Light” meme. Was he the great great grandfather of Agile or just an earlier incarnation?

** Known as an Olaf until we can find a better name.

Intent and Behaviour

A couple of year ago Ola Ellnestam invited me to visit Agical in Stockholm for the day. One of Ola’s colleagues came up with an amazing insight. When considering whether someone is in conflict or collaboration with an idea, you need to consider both “behaviour” and “intent”.

All snake oil purveying consultants have a two by two grid. This is mine…

Intent forms one axis whilst behaviour forms another.

“Intent” indicates the allignment of the individual or sub-group’s goals with the goals of the larger group.

“Behaviour” indicates how the individual or sub-group behaves towards the larger group.

An individual’s behaviour is not necessarily a true indication of their intent.

Last week, Dave Snowden said that organisations should seek out cynics because they care, whereas those who agree are simply corporate survivors. This gave me the name for one of the quadrants. When an individual’s intent is to collaborate but their behaviour appears to conflict, they are a cynic. They are one of the staunchiest allies of the group and are prepared to incur personal (social) loss for the benefit of the wider group.

I quickly made up titles for the other three quadrants (until someone comes up with a better name).

“True Collaborator” and “True Opponent” are individuals whose intent and behaviour are alligned. They can be considered to have integrity.

Cynics do not have integrity but they suffer for the common good by raising unpopular objections. Cynics prevent us from the falling into the Abalene Paradox and other similar types of group think by challenging the group’s thinking. Cynics often sport a fetching black number from De Bono’s milliners.

The most damaging quadrant are the “Snakes in the grass” . These are those individuals who behave as if they are collaborating with the group but have their own interest as their primary concern. They will collaborate while it suits them but are allways following their own agenda.

For the Agile Community, the manifesto is a call to arms to create an experiential learning community. A community that learns by doing, that test new theories in the work place.

The True Collaborator and True Opponent are fairly easy to identify. The Cynic can easily be confused with a True Opponent but a closer observation will indicate otherwise. The cynic will often have outspoken views against the group whilst at the same time maintaining strong social ties.

The hardest to spot is the “Snake in the grass”. The snake in the grass will have strong social ties with the group and will engage in collaborative behaviour with the group. When they act against the group, they will do so in secret to prevent damaging their social standing in the group from which they derive benefit. The interesting thing about the snake in the grass is that self interest will drive them in all our their social groups. If they engage in secret behaviour in one group, it is likely they will do the same in other groups. ( Liz Keogh introduced me to a great book called “Snakes in Suits” that talks about this. )

The reality is that we rarely look for enemies in our own group. Especially when we are already aware of other competitors.

So how do we spot the snakes in the grass?

1. They espouse different values to privately to those they espouse publicly.
2. They espouse one set of values publically, but then act in a different way privately.
3. They espouse integrity but then insist on hiding unfavourable information or engage in nepotism or other self interested practices.
4. They promote their own ideas when they know that others have better material.

Can you suggest other ways to spot people like that? ( People like me )

To summarise.

• You best allies may actually appear to be opponents.
• Your worst opponent may actually appear to be a collaborator.

It is important to understand the difference between behaviour and intent.

I will now don a mask, turn my back and crawl under the table as the ritual dissent begins. (Of course the great thing about ritual dissent is it gives everyone a vioce, even those with negative intent.)

Ritual Dissent

Willem has written up the CALMalpha event. This morning we did an exercise called “Ritual Dissent” whereby someone presented an idea and everyone at the table ripped into it as viscously as they could.( In real options, we call this “break the model” but do it in a nicer way. ) I intend to do just that to the CALMalpha event in the hope that future events will be better.

I did not learn anthing at the CALMalpha event. Which means for me it was a bust. A total failure. So bad in fact that I decided to leave early to pick up my kids.

The Cynefin model seems to have two major components… complexity and narrative. I named my company “Emergent Behaviour” six years ago which should tell you I had my head turned by complexity some time before that. The conversation became interesting whenever it moved in the direction of narrative. Everytime the conversation started to mention narrative, we were reminded that the material was patented and it stopped dead. Whenever someone asked about running an exercise, we were reminded this was not a training course. I was there to learn but no one was providing options to learn. I was asked to ask for sessions but the thing is, I needed an expert to guide me and present me with the learning options. There were no options. (I think this may have changed after I left)

Cynefin reminds me of DSDM before it realised it had to open source its best bits. I wondered whether the Cynefin business model was copied from the Scientologists… or whether they helped the Scientologists develop their ontologists.

I envy Willem in that he found someone who was using Cynefin. I met quite a few who had done lots of training but no one who had used it in anger.

I was looking forward to enjoying a new kind of conference organised by the expert party organisers. I deliberately maintained a relatively low key prescence. Like Willem I did not want to guide the event. The reality was I felt like a passenger in Simon’s car when he first went on the skidpan. Years ago my short story teacher told me “the reader can be confused, but the writer must never be confused. The writer should allways be in control.” I did not have a sense that the organisers were in control. We had discussions about Cynefin and Agile/Lean before some of the participants knew what they were about.I still don’t feel I know what it is about.

It was clear the organiser had an intent. They just did not want to share it. Next time, they should.

There was a mono-culture at the event that it was hard to challenge. The idea was that Agile and Lean could be improved by Cynefin. No one dared to suggest that Cynefin could be improved by Agile or Lean. Even Agile20xx has a tutorial day to bring everyone up to a base level of conversation before conference starts.

Cynefin should use fewer syllables and more common sense.

I enjoyed the event much more on twitter after I left than I did when I was attending.

Until the Cynefin team decide to share their best stuff, this CALM stuff is simply marketing to benefit cognitive-edge but marketing paid for by the attendees.

I would like to thank Simon and Karl and Joseph and Dave and Saffron and the guy in the orange jumper for attempting this. For trying something different.

A wise man once said “I’ll say that this Cynefin Lean Agile Mashup is never going to amount to a hill of beans. Please move along. Nothing to see here.” I’m gonna take his advise.