Value and Metrics

A significant risk in any business is that you do not know where you are. Value is the thing that we aim to deliver, but value data or metrics are how we measure our success at achieving our goal. The only way we know for certain that we have achieved our goal is when the value data shows us success. For example, we aim to deliver $2million from a feature. The only way we know we have achieved that is when we have the $2million in our account.

Simple right?

Nope.

  • How do we know that it is the new feature that has delivered the value?
  • Do we have to wait until the money rolls in until we know how we are doing?

If we have a single feature that people are buying it may be fairly simple, but that is rarely the case as most applications have many features. We have to put more thought into how we look at our value data. First, we need value data on how the user is using the new feature.

  • Do the users use the feature? Did the
  • How often to do they use the feature?
  • For how long do they use the feature?

Our value data impacts how we might introduce a feature to our users. Do we offer a couple of free goes before they have to pay or do they pay before they use? Offering a couple of free goes allows us to gather value data (or metrics) on how people use the feature. We want to know how long the user spends on each step of using the feature and whether they stop using the feature before they complete all the steps. This will help us understand whether the user values the output enough to put the effort in to provide all the inputs. The value data will tell us which step result in people dropping out of the process.

Context is everything. Context drives how you use value data. A web site is different to an installed application. Retail is different to enterprise. The value of each customer impacts how much we are prepared to test on them. The golden rule is to get feedback on our business case.

If we have an application that the user can download for free and have a couple of free goes before they pay, we have to build a business model ( or belief system ) around how we will make money.

For example, we assume 1,000,000 download the application. 50% use the first free go with the feature, of which 20% use the second free go, of which 10% then pay for the feature. The feature costs $10.00. This would result in revenue of 1,000,000 * 50% * 20% * 10% * $10.00 = $100,000. We could release an earlier version of the feature which tests to see how many people download and use the free versions. We can look at when people drop out of using the feature.

We aim to deliver value but the value data (or metrics) let us know whether things are going to plan.

Advertisements

About theitriskmanager

A IT programme manager specialising in delivering trading and risk management systems in Investment Banks. I achieve this by focusing on risk rather than cost. A focus on costs can lead to increased costs. View all posts by theitriskmanager

3 responses to “Value and Metrics

  • Helena

    Loved the article, as someone who plays in the IT Risk market it resonated at a lot of levels. Thanks!

  • Antoine Alberti (@antoinealberti)

    Interesting post. That’s a question I’ve often tried to deal with, without any success.
    As you said, a web site is not an installed app, and retail is not enterprise. I’ve always worked in the worst case context regarding lean startup, which is an installed app for enterprise. Which means:
    – long sales and integration cycles (thus high difficulty to map features and value)
    – few sales with big amounts (thus too few numbers to compute relevant statistics)
    – high difficulty to map usage and value to the different stakeholders (business, buyers, end users, admins, integrators, sales,…).

    If you have any clue to help me getting started, it’s a pleasure 🙂

  • timbarrassTim Barrass

    We can release an earlier version — but do we release restricted functionality? If we make a restricted release, are we learning what we think we’re learning about user drop-off, or just frustration at the restrictions? Are we likely to get feedback as well?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: