Author Archives: theitriskmanager

About theitriskmanager

A IT programme manager specialising in delivering trading and risk management systems in Investment Banks. I achieve this by focusing on risk rather than cost. A focus on costs can lead to increased costs.

IIRMFW : Exec’s & Business Value

The IT Investment Risk Management Framework is the set of constraints that development organisations need to operate within in order for the associated risks to be managed appropriately.

The executives in an organisation have a responsibility to rhe investors when managing the risk surrounding the delivery of Business Value.

Executives responsibility for Business Value Metrics

The key metric that an executive is responsible for is:

  • The percentage of the IT Investment Portfolio that can demonstrate whether an investment generated a return.

There are two sub-metrics beneath this metric for the executive.

  • The percentage of IT Investments that are linked to a metric. i.e. Has the product owner stated the metric that will improve as a result of an investment (e.g. Story, Epic, Initiative).
  • The percentage of metrics that can be displayed. i.e. Is the metric available in a format that the improvement in the metric can be demonstrated. Note that the metric might be manually captured.

In effect the executive should be able to observe a graph for each metric that is similar to the one below:

Screen Shot 2016-05-30 at 10.30.53

as well as a summary of investments that looks like the following:

MetricAllocation

In the event that a metric is not available to be displayed, the executive should be committed to delivery of that metric within an appropriate time frame.

Risks Being Managed

These key metrics demonstrate that the executive is managing key business value risks. The risks managed are as follows:

The percentage of the IT Investment Portfolio that can demonstrate whether an investment generated a return. – This metric ensures that the executive has transparency into whether a return is being delivered by their portfolio of investments. The exact return delivered by a particular investment may be difficult to assess however the impact of the portfolio investment is transparent. It is not necessary to specify that the executive checks the return regularly as the metrics graph provide transparency that is not enhanced by taking minutes of metric review meetings. However, it is to the executives benefit to be on top of their metrics by reviewing them regularly so that they can intervene in a timely manner. A failure to intervene would be evidence that the executive is not performing their role properly.

The percentage of IT Investments that are linked to a metric. – This metric ensures that everyone in the executives organisation is focused on delivering value rather than functionality. It provides transparency on those that cannot prove they intend to deliver value (no metric assigned). This transparency also allows the executive to take a portfolio view and rebalance the portfolio if appropriate.

The percentage of metrics that can be displayed. – This metric indicates where a return can and cannot be demonstrated. If the return cannot be demonstrated, there is a massive risk that an investment might deliver no return ir even worse destroy value.

The Business Value Metrics Framework

All metrics should be part of an organisation wide standardised metrics framework. This has two benefits:

  • The standard framework makes it easier for product owners to identify the value they are delivering because they do not have to work it out for themselves.
  • The standard framework makes it easier to compare the value of disparate investments.

The business value metrics framework should be “open sourced” within the organisation with a product owner facilitating the addition and modification of metrics, and the decisions regarding the additions and modifications being made by the Business Value Metrics Steering Committee. The Business Value Metrics Steering Committee should be appointed by the board as the definition of business value in an organisation is key to its success.

A business value metrics framework is vital when an organisation relies on third party consultants to write business cases as there is a substantial conflict of interest. For example, there is a tendency to suggest attributes of a product as business value (e.g. Lead time) rather than business value (Conversion Rate). Product attributes are easy to deliver compared to those that depend on the needs of the customer and the market.

The Business Value Metrics Framework can be constructed using “Break the Model”. Identify an example (Tea Bag), Reflect if it is in-scope, Create an Example (Tea Bag -> Cup of Tea -> User Need -> Business Value) and then Model (Add it to the Hierarchy). The product owner performs this process based on the examples that are presented to them as not fitting in the model.

The Business Value Metrics Framework will probably map to your organisations Business Model. e.g. Freemium = Get users, Increase Activity, Get Revenue. Traditional = Get Revenue, Increase Activity, Prevent Churn.

Risk Adjusted Return on Capital is a good starting point.

  • Risk
    • Investment Lead Time (Weighted Lead Time)
      • Key Man Dependencies
    • Quality
      • ITIL measures
      • Failure Demand
    • Employee Happiness
      • Turnover
  • Revenue
    • ARPU (Average Revenue Per User)
    • Activity (Average Visits per Month)
    • Number of Customers
      • New Customers
        • Conversion Rate
        • Customers into the Funnel
      • Churn
  • Costs
    • ACPU (Average Cost Per User)
      • People Costs
      • Overheads

Disruptive Innovation versus Efficiency Innovation

The executive should ensure that the portfolio has the appropriate levels of Disruptive, Sustaining and Efficiency Innovation. Otherwise the organisation might discover its market being disrupted. Check out Clayton Christensen’s lecture for more details.

* The concept of the Business Value Cascade was created by Mark Gillett, Senior Vice President at Microsoft, responsible for Skype & Lync products. The first analysis and implementation was performed by Keith Beatie, now a partner at McKinsey.


IT Investment Risk Framework and Software Frameworks

An IT Investment Risk Framework is a set of constraints that an organisation imposes on its development organisation to ensure that IT Investments are delivered in a way that manages the risk involved. It should create failure containers so that any failure does not propagate across the organisation like the failure at Knight Capital which caused the organisation to fail.

The boundaries imposed by the framework should be negotiable rather than fixed, otherwise the framework may fail catastrophically. (Hat tip to Dave Snowden’s Cynefin Framework.)

The Risk Framework should NOT specify how the IT Investment is made, simply the way that IT Investment risk is managed. The Risk Framework specifies a number of Commitments placed on the development organisation when they accept funding for an investment.

By comparison SAFE and LESS are frameworks that seek to optimise the delivery of value to the organisation. They provide a number of enabling constraints in the form of principles. The SAFE and LESS frameworks can both be deployed within a IT Investment Risk Management Framework providing they satisfy the its constraints. In effect, they are an Option that the development company may adopt.

In summary an IT Investment Risk Management Framework is a commitment placed on a development organisation whereas the SAFE and LESS frameworks are options available to the aid development.


An Agile Risk Management Framework

I love working with smart people. They make it easy and fun to solve nasty problems. Tony Grout is one such person. He is a really deep thinker on the subject of theory of constraints. One of the problems faced by many large organisations is that they have a prescriptive software development life cycle or SDLC for short. The organisation insists that all IT Projects follow the SDLC. The SDLC is normally very binary in nature. If your project is of type XYZ, then you need to do ABC. This poses a problem for Agile development which doesn’t follow the same philosophy. Rather than one size fits all, Agile development adapt to the risk profile of the context they are operating in. Often without realising it, most Agile development follows Alistair Cockburn’s Crystal methodology.

At Lean Kanban London Day, Tony Grout and I presented our view on the Agile Organisation. (Hat tip to SAFE).

AgileOrganisation

A key part of that organisation for any Large IT Organisations is the Governance Function. This consists of four key parts (A metaphor of speed restrictions for drivers is provided in brackets to aid understanding):

  1. An IT investment risk management framework. (The law stating 30 mph in residential areas. 20 mph is areas with people at heightened risk such as schools and hospitals)
  2. A policing* function to ensure that all investments operate within the framework. (Speed cameras and traffic police)
  3. A governance function to ensure the risk management framework evolves appropriately according to context. This function also acts to help interpret the framework in a particular context. (Courts and law making bodies)
  4. The responsibilities of ordinary citizens. (Driving tests).

Separate to the governance function is the capabilities that allow development teams to deliver IT investments. The problem with many SDLCs is that they combine the risk management framework with the capabilities needed to deliver them. The IT investment risk management framework should be the smallest set of constraints that the organisation imposes on the development and change organisation to ensure that the risks associated with the IT investment are managed appropriately. The constraints should be helpful and instructive rather than arbitrary. The constraints should give comfort to investors that their investment will be protected against the three categories** of risk, namely:

  1. Delivery Risk – Will the functionality be delivered.
  2. Business Case Risk – The IT Investment delivering the stated return.
  3. Existing Business Model Risk – Will the IT Investment damage the existing business model.

Tony and I categorised the risk management framework into the following attributes of an Agile development. If an IT Investment does not satisfy these criteria, it should be considered NOT Agile and thus managed under the existing SDLC. These categories are:

FourRiskGroups

We even use them as part of our training.

LKD Manifesto

The policing function is particularly important for an Agile IT Investment Risk Management Framework as context is so important to the approach, and it is easy to game the rules. The policing function should work closely with the organisation’s Agile Coaches to help development teams learn to operate within the rules. This policing function should never be performed by Agile Coaches as it would destroy the relationship between the coaches and the development teams.

The IT Investment Risk Framework specifies the impact of each risk category on each part of the organisation.

ITRiskFramework.png

Our next posts will expand on the framework, starting with the detail for “Delivering Value” for the “Exec”.

* By policing function, I am referring to one like an idealised 1950s Britain where the police are considered public servants (Dixon of Dock Green) rather than some countries where they are considered a domestic army to suppress the people.

** These three risks were first articulated by Steve Freeman and myself in an article published in the Agile Alliance’s Agile Times in 2005(Date ?).


Managing the Top 2 constraints in an organisation.

If you ask most people what the constraint is in an organisation, they will tell you that it is the budget. You can tell this from the amount of energy that companies put into managing and controlling budgets and funding. Much of the upper management spend all of their time and effort controlling budgets.

However budget is not really the constraint. Try this thought experiment. Imagine you have infinite budget. Imagine you have all the funding that you require. Imagine that your company has been bought by an infinitely rich benefactor who says “money is no object”, and means it. What is the constraint now?

Queues form in front of constraints when demand exceeds capacity. This lack of capacity at constraints results in work items being delayed. Constraints reduce the queues in front of downstream processes, and those downstream processes potentially run out of work to do. These downstream processes hungry for work will generate work to keep them busy. The net result is that increasing budget results in more work in progress. Increasing budget does not necessarily increase output from the system… much to the frustration of those funding everything.

The first constraint is the capacity of each teams to satisfy demand.

It takes time to shift the capacity of an organisation. Small corrections in capacity involving the movement of staff or work and typically take a couple of months. Increasing the capacity of the organisation which involves hiring new staff can take a few months. Capacity is only increased when the person joining a team is competent. A new person occupying a role is initially a drain on capacity.

The second constraint is the capacity of individuals within the team.

The capacity of a team is a function of the constraints within the team. That is, the capacity of the team is limited by the capacity constraints of the individuals in the team. If the demand for a certain skill exceeds the capacity of the individuals in the team with that skill, then a queue will effectively form in front of the skill capacity and those working behind the constraint will be starved of work and take on lower value work. Even in cross functional feature teams, it takes time for a new member of the team to come up to speed.

Managing the Capacity of Teams in an Organisation

Tony Grout and I developed Demand Mapping at Skype with a large group of collaborators including Lisa Long, Ram Rao, Marina Oliveiro and John Horton. At the same time Dan was developing Delivery Mapping with his clients. This became apparent when Dan North bought me in to one of his clients to introduce it there.

Demand Mapping starts with the understanding that the constraint in organisation is the teams (or scare resource such as servers / server space) rather than the budget. The goal is to create a backlog that optimises the value that can be generated from the (currently) fixed capacity of teams. A secondary goal is to identify constrained teams with no capacity and teams with spare capacity.

Consider a team that consists of four cross function scrum teams that all maintain and develop “Component X”. For the next three months, that team would have twenty four team weeks of capacity ( Four teams times twelve weeks times 50%* ) to work on “Component X”.

We have five initiatives that require the capacity of the “Component X” team. We order the initiatives in terms of the value we expect** them to deliver:

  • Initiative 1 requires 100% of the capacity
  • Initiative 2 requires 50% of the capacity
  • Initiative 3 requires 25% of the capacity
  • Initiative 4 requires 100% of the capacity
  • Initiative 5 requires 25% of the capacity

We can now choose between the scenarios of Initiative 1 only, Initiative 4 only or Initiatives 2, 3, and 5.

We repeat this for all of the teams, creating a portfolio that optimises the delivery of value.

We are left with the portfolio of initiatives for the organisation to deliver, and the capacity utilisation of each team. The teams deliver the first whilst management works to re-balance the second using the tornado map (See Todd Little’s risk presentations) to determine future demand on the teams.

Managing the capacity of individuals within the team.

Rohit Darji and I developed Staff Liquidity and the Skills Matrix a couple of years before I discovered Agile. Dan North took these tools and evolved them into the more elegant and useful Skills Mapping***. He extended the values that individuals self score themselves on to the following.

  1. My current skill level.
  2. The skill level other members of the team would say I have (Moral Hazard).
  3. The skill level I want to have.

Ideally the team then self organises to remove key man dependencies and cross skill to remove constraints within the team. Unfortunately some organisational cultures mean that management need to intervene in order to ensure that skills transfer occurs.

To summarise, manage constraints caused by teams and within teams. Acknowledge that the budget is rarely the constraint. Thats why you need Demand Mapping and Skills Mapping in your tool kit.

My thanks to Joshua Arnold whose post “Resources are the constraint” inspired me to write this.. initially as a comment responding to his post.

* The maximum capacity a team should allow is 50% otherwise queues will naturally build up.

** Expect is probabilistic term. A summary of the range of value based on the probability of them reaching that value. Normally we use a “HIPPO” version of the expectation of value rather than use a formula such as Black Scholes to calculate the value of the option.

*** Check out Dan’s talk at YOW! for a fantastic introduction to Skills Mapping and the rest of Business Mapping… Demand Mapping and Initiative Mapping.

 


SAFE and the 1%

This is a response to the discussion with Martin Burns on Joshua Arnold’s blog about WSJF and SAFE. Martin tweeted “real SAFe talk is Program & up. By defn only 1% of people impacted”

And there you have it. By definition, According to SAFE, only 1% of people are involved in Program and Portfolio level.

Tony Grout and I worked with a large number of people at Skype to create something that Dan North and I now call “Delivery Mapping” (Part of “Business Mapping”). It is a collaborative framework that allows large or small groups of product owners to collaborate on the creation of an organisational level backlog. Rather than tell people what to do it provides some minimal constraints in order to force collaboration and ensure that the right conversations are happening. The outcomes of “Delivery Mapping” are a strategic organisational level backlog based on the team level constraints, and a map showing which teams are constraints, or have no work on the strategic backlog (i.e. Team level utilisation).

To create the organisational backlog, every product owner was involved, along with all of product owner management team. The ideal sweet spot was that tech leads and test leads worked with the product owner in this process. This means that about 20% of the organisation collaborated to create the backlog. Even more are needed to address the team rebalancing effort. It meant that a significant portion of the organisation were involved in strategic decisions, all playing a small part and aware of the bigger picture. Compare that to SAFE where only 1% of the organisation are expected to make all the decisions.

I actually advocate SAFE. They have an interesting story at the program level with the Agile Release Train (ART). It exists on a continuum at the extreme end to LeSS with its self organised feature team creation. That said, the fact that SAFE thinks only 1% of people are needed for the Portfolio and Program indicates that its creators (SAI) do not appreciate that the constraint exists at the team level. They still think that the constraint is the budget (with associated plug compatible programming units). It also indicates that they think some elite group should be solely responsible for creating the portfolio. Our experience shows otherwise.


MVP considered harmful. Introducing the MVI.

When adopting Agile at scale, we always need to remember the three things that have the most impact… Context, Context, and Context.

MVP or Minimum Viable Product is a fantastic idea for start ups. It tells you to build the smallest, most minimal product that will be viable for your target customer segment or market. MVP is about discovering a market for a new product. Interestingly many people miss the important aspect of MVPs. MVPs are a risk management approach to help you determine whether your hypotheses about your target customer segment and their needs (or jobs to be done) are correct or not. MVPs are nothing to do with your product really. They are to do with the customer segment and their needs. A start up is defined as a organisation that is establishing its customer segment and the customer’s needs.

Normally when we talk about scaling Agile, we are talking about introducing Agile to an organisation with established products and markets. And this is the problem. These companies already have customers. These companies already have market share. These companies already have products that are more than minimally viable. As they engage in an Agile transformation, they adopt Agile, Lean, Lean UX and Lean Startup concepts… including MVP. However they already have a product so they assume that MVP means the MVR or “Minimum Viable Rewrite” as they transform to Agile and completely replace existing products with “Agile” products. MVP becomes the minimum amount of the existing product that needs to be recreated in the new “Agile” product. That minimum tends to be a complete rewrite of the existing product. In other words, a completely non Agile approach to transformation.

Over the past few years I have seen a pattern emerging in organisations that have mature products as they adopt Agile. They all have a flagship Agile project that has a scarily familiar description. They have successfully implemented “Scrum” (This is in no way a criticism of Scrum because they are not really doing Scrum). They proudly tell everyone about their two week sprints and monthly/bi-monthly releases. The problem is that the releases are always into a “testing” / “staging” / “pseudo production” environment. The flagship project normally has one or two years worth of inventory in the non production environment and they are about to release a huge change on the business. The only risk this approach addresses is the risk that “IT” will deliver into that environment. There is no feedback from the market that the rewrite product is fit for purpose. In effect the risk that the business case (understanding of the target market’s needs) is wrong or the damage to the existing product (market share) are ignored.

The alternative for organisations with mature products is to consider an MVI or Minimum Viable Investment approach. First instrument up your product so that you understand your current metrics. Then identify investments into your product to improve it. Gradually make small investments into your product until you can make big architectural changes easily. In effect you refactor your product towards a new vision of the product.Start from the outside and work your way in. In other words, start with your customer and work back through the value stream. Understand that taking an inside-out approach will lead to a massive and risky transformation that ignores your customer.

Transform your approach to your product rather than try to transform your product itself. Transforming your product is normally a good indicator that you have not transformed your approach and thinking. It is big batch change.

To summarise:

If you are building a new product for a new market, use an MVP approach.

If you have an existing product (i.e. market), adopt an MVI approach.

If your transformation partner (consultancy) is advocating a big bang approach to transformation, sack them. They do not understand Agile and should be cast out and shunned for selling snake oil. Obviously they wont say its a big bang approach, but measure the lead time to production customers. Let the data reveal reality rather than opinions.

 

 


One Organisational Backlog? Or Two?

I have been working with a group on whether they should have a single organizational backlog or whether they should separate their backlog into a product backlog and a technical improvements backlog. Like many places they want to make sure that infrastructure and technical debt is prioritized properly.

Any practitioner will tell you that technical improvements generally get ignored in favour of product improvements. Technical debt pay down normally gets deprioritized when there is a chance to improve the product from the customer’s perspective. At one organisation I worked with, the CEO said that thirty percent of every team’s capacity should be focused on improving the quality of the product. Every quarter the CEO would review where the teams were focusing and discover that product improvements dominated, normally taking one hundred percent of the effort.

In theory a separate backlog for technical improvements makes sense. We assign a certain percentage of capacity of each team to technical improvements and then we work through the technical improvement in the order in the backlog. However, in practice, we know that the teams will work almost exclusively against the product backlog. When you have two backlogs, you have the problem of deciding how the priority of each item in the technical improvement backlog relates to the priority of items in the product backlog.

The only solution is to have one backlog containing both Product and Technical Improvements. The investor group that prioritises the backlog considers the relative importance of each item regardless of whether it’s a product improvement (i.e. customer / business benefit) or a technical improvement. That way it is clear to teams that the technical improvement is more important than that new feature that the client wants.

So where does the confusion in Agile circles come from? It turns out that SAFE advocates the use of two organizational backlogs, a product backlog and an architectural backlog. The authors of the SAFE framework must know that two backlogs create this problem that leads to excessive technical debt, so why do they advocate this approach. My theory is quite simple. SAFE advocates the use of a particularly bad implementation of Weighted Shortest Job First (WSJF) to prioritise the product backlog. WSJF contains a bias that favours backlog items where the outcome is known. In Cynefin terms, WSJF has a bias towards “Obvious” and “Complicated” items, and against “Complex” and “Chaos” items. Technical backlog items often (though not always) fall into the “Complex” and “Chaos” domain. My theory is that the authors of SAFE have seen this problem in early implementations of SAFE and so separated out the Product and Technical Backlog.

Practice has shown that one backlog is needed if technical items are going to be given the appropriate level of priority. So how do we do this in practice? The real problem is that technical improvements are often expressed in terms of cost and the “What” / “How”. Technical improvements are rarely expressed in terms of the benefit they will deliver. To do this, two additional organisational level metrics are required, one for customer perceived quality (functional bugs, performance bugs, UX bugs, availability etc), and one for the lead time to deliver value (e.g. weighted lead time for investments and lead time from detection to fix for bugs). Technical improvements can be expressed in terms of these metrics (e.g. Paying down this technical debt will reduce the uncertainty of lead time for this change to the component, or this item will reduce the probability of bugs). The outcome is often unknowable which means they are in the “Complex” or “Chaos” domain, however the investor group can understand the intent. Once the intent is known, it is possible to construct a narrative that explains the value of the technical item in the context of other product investments. It is then possible to construct a backlog where WSJF may assist the prioritization discussion but does not dominate it.

Another couple of related points.

  1. Paying down technical debt is a great way to train new people on a code base and reduce key man dependency (a form of technical debt). If the organisation knows it will be making major changes to a particular component, investing in the pay down of technical debt is a great way to prepare for that future development and build additional capacity so that it can be done quicker.

 

  1. A more important point is that having to put technical debt into the organizational backlog is a transitional state. It is necessary because the teams allow the build up of technical debt and see the need to pay down debt in large chunks. In mature Agile teams, the teams will gradually improve the quality of the code base as part of every piece of work that they do. For example, a few years ago I visited Nat Pryce on a project. He showed a graph that his team had created. The graph showed the number of lines of code in their application. When they started it contained one million lines of code. After six months it consisted of one hundred thousand lines of code. They had not taken time out to pay down of technical debt, rather it was a continual process alongside developing new features. On a mature team you are less likely to see backlog items to clean up technical debt because it is a continual process that is part of normal development. In other words, a mature Agile culture will clean up as they go along whereas a immature Agile culture will have teams that see a choice between delivering features and creating technical debt. It would appear that SAFE intends to embed this immature practice in its process by having a separate technical backlog.

In conclusion, a separate technical backlog is a failure state. A technical backlog institutionalizes immature practices and creates a separation between product and technical concerns when there should be no split. A second technical backlog hides technical concerns from the product organisation when they should be a primary concern. Instead, create Quality and Lead Time based metrics that allow engineers to communicate the importance of the work they need to do.

One backlog to rule them all, not two or three or more!


Follow

Get every new post delivered to your Inbox.

Join 98 other followers