Author Archives: theitriskmanager

About theitriskmanager

A IT programme manager specialising in delivering trading and risk management systems in Investment Banks. I achieve this by focusing on risk rather than cost. A focus on costs can lead to increased costs.

Risk Aversion and Risk Management – A case study.

WARNING: I’ve been lazy. It is quicker and easier ( and more fun ) to write this than hunt through the inter-web to find out if someone else has already written this down (I’m kinda assuming Troy or Liz have). I just need something to refer to.

I have been helping a waterfall Scrum* project adopt some Agile practices. Unfortunately the project had reached the point where there was a large backlog of manual User Acceptance Testing to be done. During the week I had a great conversation with the test manager for the project (The person producing the management reports). I realised that it was the perfect case study to understand the impact of the culture (risk aversion versus risk management) on a project.

The test manager categories all bugs as either Priority 1 or Priority 2. All of the Priority 1 & 2 bugs need to be fixed and signed off for the UAT to come to an end. Note that it is not desirable to time box the UAT on a Waterfall project as this will drive the wrong behaviours. The inspiration for this post came from a conversation on whether Priority 1 or 2 bugs should be fixed first.

Managing the UAT

In order to manage the UAT, the test manager needs two graphs.

  1. A burn-down of the tests to be run.
  2. A Cumulative Flow Diagram showing bugs as they are opened and closed.

The graphs below illustrate this point.

new doc 4_1

The UAT is over when the number of bugs closed line (Green) intersects with the number of bugs open line (black), assuming the zero outstanding test cases which have not reached the end of the test cycle.

There are two key milestones for a UAT.

  1. The most important milestone is reaching the end of the UAT (Obviously).
  2. The second important milestone is when all test cases have reached the end of their cycle, even if they contain bugs and are not signed off. The burn down for this is the blue dotted line. The reason that this is so important is that this represents the point at which further bugs are unlikely to be raised. The bugs discovered after this point are likely to be introduced as a result of fixing another bug.

The impact of Culture

So what does this have to do with culture?

In a risk averse culture, we avoid or ignore risk. We would want to demonstrate our competence as soon as possible. This would means we would want to demonstrate progress as soon as possible to gain the confidence of our managers.

In a risk managed culture, we would want to address the riskiest elements of the work first. We would be more interested in doing what is right for the organisation than demonstrating our own competence quickly.

Opening Bugs

So how would this impact the testing cycles?

If we are a risk averse tester we would want to demonstrate progress. We would want to show that we were getting through the test cases as quickly as possible.

Our focus would be on the pink line as this shows progress to the end of the UAT, the most important milestone.

To do this, we would pick those test cases that are easiest to sign off. We would pick those test cases that are least likely to contain bugs because they are quick and easy.

Furthermore, we would want to close bugs and sign-off test cases in preference to finding new bugs.

A risk managing tester would want to find bugs as soon as possible. Our focus would be on the blue dotted line. We would want to get to the end of all the test cases so that we would know most of the bugs in the system. For us, the most important thing would be to see the number of bugs raised flatten off as this allows us to manage the fixing of the bugs.

The graphs below demonstrate the impact of culture on the shapes of the lines. It is assumed that the same project with identical number of bugs discovered.

new doc 6_1

Both projects finish on the same date because the contain the same amount of bugs that require the same amount of effort to fix. However, consider the perspective of the test manager. In the risk managed culture, they know that they have discovered most of the bugs much earlier, and can take appropriate resourcing decisions.

Closing Bugs

So how would this impact the fixing of bugs?

If we are a risk averse developer we would want to demonstrate progress. We would want to show that we were getting through the bugs as quickly as possible.

Our focus would be on the green and pink line as this shows progress to the end of the UAT, the most important milestone.

To do this, we would pick those bugs that are easiest and quickest to fix and sign off. We would defer bugs that are complex, or take a long while to fix. We would potentially defer those bugs that block us reaching the end of the test cycles if they are complex or take a long time to fix.

Furthermore, we would want to close bugs and sign-off test cases in preference to finding new bugs as this looks better on management reports.

A risk managing developer would want to find bugs as soon as possible. Our focus would be on the blue dotted line (fixing bugs that block us from reaching the end of test cycle), and on fixing bugs that are most likely to result in further bugs, namely the complex and large bugs. The number of bugs closed and the number of test cases signed off would be of secondary importance.

Assuming a risk managed tester, the below graph shows the impact of culture that the developer works in impacts on how bugs are fixed.

new doc 8_1

The risk averse developer would want to work on easier bugs that show them making progress. As a result there might be a delay (marked as “A” on the graph) on finding all the bugs in the system. This means that the risk managed developer would know with more certainty how much work needs to be done for “A” units of time.

The risk managed developer trades uncertainty at the start of the testing cycle for more certainty at the end of the test cycle. Conversely, the risk averse developer pushes more uncertainty (risk) to the end of the test cycle.

The risk averse behaviour makes it very difficult to manage resource loading for the test cycle whereas risk managed behaviour allows effective management of resource loading.

Simply put, risk managed behaviour gives more options than risk averse behaviour.

A final point on this, consider the projection (using a velocity measure) of the end of UAT. The graph below shows that the risk averse behaviour results in larger and larger time extensions at the end of the testing cycle ( A -> B ), whereas the risk managed behaviour results in smaller and smaller reductions of the time to the end ( A -> B ).

new doc 7_1

In conclusion, risk managed behaviour does allow management of testing, whereas risk averse behaviour does not. Anyone managing a testing cycle should understand that attempting to demonstrate early progress leads to a loss of management and control.

*These days it is really easy to find Waterfall projects. They are normally called a “Flagship Agile” project. The irony escapes them. A Culture Changing Agile Project would be called an “Agile Canoe” or “Agile Raft” or “Agile Rowing Boat”. Certainly not the flagship which is normally an aircraft carrier.


The Responsive Manifesto

I am increasing working with groups other than IT who want to adopt Agile. In addition, Agile is becoming an increasing toxic term causing division and fear within the organisations that I work. On that basis, as a bit of fun last week, Richard Warner and I wrote our own “Responsive Manifestos”. Both were very similar. This is mine.

We chose the word responsive because it is a term that has resonance with the business. There is already a concept of responsive marketing. Responsiveness is the goal, agility is the way that IT has chosen to achieve the goal of responsiveness.

By choosing to break with Agility, it allows the business to focus on the goal rather than the implementation of Scrum, Kanban or Lean as a end point for Agility.

Screen Shot 2015-03-15 at 14.32.41


PM/UX/Design/Data/Agile Pulling it all together

For the past couple of months, Richard Warner and I have been working with a number of teams to help them better work together. Each of the teams contains a Product Manager, UX Researcher(s), UX Designer(s), and Big Data Analysts who are working in an Agile manner.

Last week Richard and I sat down to work through one of the problems we are facing. A number of teams are using some kind of “Agile Canvas” derived from the start up community. It was our feeling that these canvases are driving the wrong behaviour in the teams. The most popular canvas with our teams is the one taught by Roman Pilcher.

Screen Shot 2015-03-14 at 10.06.26

You start with a strategic statement of intent. Then you list the Target Group (Customer Segment/Market), Needs, Product, and finally value. As Richard and I were chewing this over, we realised that although this was a great canvas for a start up, it lead to the wrong behaviour for enterprises. Value is an after thought. For a start up, you establish your product in the market and then think about revenue models, which often means being bought out by another more established company (The Instagram approach). As an established enterprise, this is a risky approach as you may develop a product that is not sustainable. If you pull the product, your customer will stop trusting you which will impact your existing business. At this point we decided to use “Break the Model” from Feature Injection to model the different order in which you perform the tasks. We realised that there are a number of different valid paths you would take based on the nature of the insight driving your discovery process.

IMG_20150310_164516927IMG_20150310_164545172

We realised that a design hypothesis that we test potentially contains a number of sub-hypotheses.

IMG_20150310_164530515

A design hypothesis needs to contain each of these sub-hypotheses. However the order in which you build up the design does not matter as long as the Feature/Design is the last thing you consider.

IMG_20150310_164613391

Richard and I looked at all the different combinations of order in which you could create the design hypothesis.

Screen Shot 2015-03-14 at 10.56.04

So here is our working hypothesis. Depending on your insight, you need first to establish the outcome, market and needs. Once you have these three, they form the design brief for the UX designer to design the “feature”.

The feature should always be the last part of the hypothesis. If you start with a “request for a feature”, this should be regarded as an insight (tea-bag). If we end with the outcome, customer or needs, they will likely be ignored. If you end with the outcome, you may be subject to reputational risk if you have to pull the feature because it is not sustainable (not a problem for startups who are looking for an IPO exit). If you end with the customer, you will not know who should like the feature and who should not. You may lose an important customer segment without realising it. If you end with the need, you may deliver a product that nobody wants, like Segway or Betamax. Betamax was higher quality but did not satisfy the need of fitting a movie on a single tape.

Screen Shot 2015-03-14 at 11.16.28

The PM is responsible for establishing the correct outcome. Data Analysts and UX Research are responsible for providing insights and customer segmentation (market). The PM/UX Rearcher/Data Analyst and UX Designer “pair” to identify the customer segment and associated insight that is most likely to deliver the outcome. From the insights, they create hypotheses about the customer needs to be satisfied.  Note that sometimes, the process starts with an outcome, sometimes with a need, and sometimes with a customer segment. It should never start with feature. If the starting point is a request for a feature, this should be treated as an insight.

IMG_00013

The combination of Outcome, Customer Segment (Persona) and Needs form the design brief. e.g. In order to increase blah, as a blah, my need is blah. ( A Job Story ). There may be several job stories. e.g. The need to learn quickly at the expenses of a slower process, and the need to learn slower in order to have a faster process. Each design brief may result in several designs.

Screen Shot 2015-03-14 at 11.14.39

It is the responsibility of the PM and UX Researcher to confirm whether the designs have achieved the design brief. The designs can then go through Melissa Perri’s “Bad Idea Terminator” process. We start with the fastest and cheapest ways to invalidate a hypothesis, gradually moving on to more expensive methods. (e.g. Team/Expert review, Usability Testing, then MVT, and finally full production)

Screen Shot 2015-03-14 at 11.14.53

As soon as we get a UX design, we can create the Given-When-Then scenarios to describe it’s behaviour.

Screen Shot 2015-03-14 at 11.13.35

This process will help us split the design (Epic) into several Epics. Once we have the Epics with GWT scenarios, we can easily slice them into user stories.

Screen Shot 2015-03-14 at 11.18.07

At this point it is pretty easy to identify the APIs (services) needed to deliver the application.

This whole process is summarised as follows:

Screen Shot 2015-03-14 at 11.19.57

Richard and I are looking for feedback on where we got it wrong. :-)


Managers and Management

My thanks to Marc Burgauer for spotting the discrepancy between my meaning of the words manager/leader and the common understanding. A true act of management. This blog is a pretty much straight up copy of the presentation I gave at the APLN in Autumn 2009. IT is based on this article published on InfoQ.

I have a problem with Leaders and Managers. I have a problem with the idea of anointed / appointed individuals being in control just because they were trusted rather than competent. I own it as my own problem as others do not have this problem.

To me, A leader is someone who engages in acts of leadership. A manager is someone who engages in acts of management. Leader is not a role or job title. Manager is not a role or job title.

  • Leaders are the guardians of the future.
  • Managers are guardian of the present (or rather the current processes).

So what does a Leader do? What are the Leadership tasks that are needed for a healthy organisation?

  1. They are responsible for the visionsssss. <– Note plural. Leaders ensure an organisation has options. Unlike leadership at Nokia that killed options for the future and left the company unprepared for failure.
  2. They prepare the organisation for the unknown. The classic tool here is scenario planning. Talking through scenarios so that the organisation can react when faced with unexpected events.
  3. They look for new risk and opportunities. They are scanning the environment for options.
  4. They give us time to react. By making us aware of these options, we have time to prepare.

So what does a manager do? What are the management tasks that are needed for a healthy organisation?

  1. They facilitate the creation of the process. They don’t own the process. They own the responsibility for the process. To ensure that it is healthy.
  2. They watch out for broken processes. They look for risk in the process that is unmanaged.
  3. They look out for gaming. Those that cheat in a transparent way so that the weaknesses of the process can be managed.
  4. They monitor and manage the known risks and opportunities.

The punch line of the presentation is…

We need fewer leaders.

Screen Shot 2015-02-22 at 10.29.25Screen Shot 2015-02-22 at 10.29.43Thats why I love the work Dan Mezick has been doing using Open Space as a tool to create communities of Leaders and Managers within organisations that want to change.


Risk Averse & Risk Managed

In Seeing Culture, I introduced the idea that a culture is “Risk Averse” or “Risk Managed”. It has been pointed out to me that I did not explain what I meant by these terms. I come from an investment banking background and that’s where my understanding comes from. Please shout out if the terms I use are unclear, it’s probably because of the finance language and that I never went to a public school to learn the propa meanin of werds.

Risk exists. Regardless of whether we are risk averse or risk managing, the risk is there. The difference is our approach to risk.

Risk Averse means that we attempt to ignore risk or get someone else to own it for us. Often risk averse individuals will feel powerless about risk and will attempt to hide it or deny that it exists. Risk Averse refers to our intent rather than our behaviour. The irony is that people who intend to be risk averse engage in very risk behaviour. When risk averse individuals take risks, they gamble. They make big bets and they take a punt. They ignore the risk and hope that everything will turn out OK. Sometimes it does, often it doesn’t. Managers get promoted because they are lucky. In fact, in a risk averse culture, the way to get ahead is do nothing and wait for those who do try something special to fail.

Risk Managed means that we manage the risks around us. There are many strategies we can adopt to manage our risk, which include:

  1. Do nothing.
  2. Monitor the risk.
  3. Make the risk transparent.
  4. Hedge the risk.
  5. Get an expert to manage the risk.
  6. Sell the risk.
  7. Buy insurance.

The point is that a risk managed culture attempts to make risk transparent and visible so that it is managed, whereas a risk averse culture will attempt to hide risk, ignore risk, or off lay it on someone else. Risk managed cultures attempt to understand risk so that they can manage it.

I recently ran a session on the role of the IT Project Manager (They are there to manage risk funnily enough). We ran an exercise where each group would pick a process point, and identify the risk it was meant to mitigate. Then they could suggest an alternative process to manage the risk. One group picked “Production Sign-off”. This was where the team got senior management to sign off that the software was fit to go into production. When we discussed what risk it mitigated, the answer was “The manager will get sacked instead of us in the event of a serious fail in production” The reality is that the manager would still sack you and is unlikely to be sacked themselves. The risk they are trying to address is a complex one “Failure in Production” and they are trying to address it with a Complicated Strategy, by going to an “Expert”, “Hippo” or “Authority” to cover themselves in the case of failure. They are NOT mitigating the risk of failure, only the impact on them…. And it does not do a good job of that either. The real risk was “How do we handle a production fail after release?” This is an outside context problem as we do not know what might cause the failure. We were then able to start developing options to manage the risk. A “Roll back strategy”, “Monitoring and failure detection”, “Phased roll out”. In effect, we need to develop systems that are resilient and robust. Sign offs are a sure sign that people are being risk averse.

Years ago I worked for a large multi-national. It came up that they did not insure employee’s laptops. “Why not I asked?” “It cost more to insure them than it does to replace the ones that are lost/stolen/broken” The risk was not ignored, it was quantified and the company decided it was cheapest to do nothing. This is why “Catastrophe Insure” exists. Some companies find it cheaper to not insure business as usual risk, but they do insure against existential risks. This was a sign that the organisation was risk managed.

The interesting thing about risk is that we can have a different risk preference for different types of risk. A company may be amazing at managing its business risk but then have a “Vendor assumes the risk” attitude to IT investments. They are blind to the fact that the vendor does not assume all the risk, and they are ignoring significant risk. Risk aversion often has to do with our understanding of a subject and the risk associated.

Most people are risk averse when it comes to their financial investments related to their retirement. They consider finance to be a scary and risky world. As such they will tell their financial adviser to invest in the safest or least risky investments. They will then gamble or have a flutter with some small proportion of their investments. So most of their pension goes into investments (Government Bonds) that will provide a return just above or just below inflation. A small proportion will then be invested in a start up. A more balanced approach might see someone investing in an equity pension. The great thing about equity pensions is that they go up when the market goes up, and they go down when the market goes down. People check their pension performance once a year when they get their statement. Most people investing in a pension do not know the track record of their financial adviser, and they do not know the name of the person managing their pension fund.

Risk takers invest in hedge funds. They know the name of the trader running the hedge fund. They probably know the trader personally. In fact, they were probably the trader’s manager when the trader worked for an investment bank learning their trade. Each day they will check the price and risk profile of the fund. The hedge fund will be expected to make money when the market goes up AND when the market goes down. The investor will expect an appropriate level of return for the level of risk they are taking. They do not ignore risk, they learn about it.

So here is the punch line. ALL cultures should be risk managed. There is no excuse for being risk averse. As companies, the risk most people are concerned with is “Will I be fired?” as they think the company is big enough to survive on its own. Most employees do not think “Will the company go bankrupt?”. The reality is that large companies now go bankrupt. Fifteen years ago investment bank made money out of “over-supply” in a market through juicy merger and acquisitions fees. Companies did not go bankrupt because banks did not make money out of bankruptcy. The problem with M&A revenues was that they were lumpy and were not predictable. Fifteen years ago an innovation called Credit Derivatives came into being. Investors pay Investment Banks an insurance premium to protect them when companies go bankrupt. Now investment banks can make money out of bankruptcy. Not only that, but is a steady predictable income stream. A few bankruptcies now and then are good for business.

It also explains why smaller companies tend to have a culture that manages risk. The biggest threat to your salary is that the company goes bankrupt.

Sorry its a long blog post, I did not have the time to write a short one.


Culture and Needs

Thank you to Bob Marshall for inspiring me to write about needs with his work on the Anti-Matter principle. I feel a sense of sorrow when I read the anti-matter posts as I get a strong sense that Bobs needs have not been met. Also, that he may not have met the needs of others and that satisfying the needs of others might be a bit of a revelation to him. Its a hypothesis and I would be delighted if someone could show I was wrong.

When we go to work I feel that there are several different groups who’s needs need to be met. In no particular order, those groups are:

  • Employees (my colleagues) from the CEO all the way down
  • Employee’s family and friends.
  • Customers.
  • Providers and Suppliers.
  • Investors in the organisation.
  • The community in which the organisation exists.
  • The environment.

In order for an organisation to have a long term healthy and viable future, all of these needs have to be in balance.

For customers, the focus is on user experience, but sadly that is often only the case in situations where users have choice. In user experience, we talk about “user needs”, “jobs to be done” and “job stories” to reflect needs. Some much loved companies focus on their customers at the expense of other groups. For example Apple is famous for being focused on the needs of its customers but not the needs of its employees (especially those working for suppliers in China).

Some organisations focus on the needs of the “shiny” employees. Classic examples include Hollywood Studios, Premier League Football and Investment Banking. This is at the expense of the Customer, other Employees, Investors and the Community.

The needs of the organisation (investors) are often expressed in the form of business value.

My point is that needs are quite well understood. Its just we don’t just always call them needs.

Quite often, when we talk about needs, the group we feel gets ignored is the employee. And normally we mean ourselves.

In a risk averse culture, the organisation and its management/leadership will quite happily ignore risk. This is often evidenced because they ignore the needs of one or more groups. In a risk managed culture, the organisation and its management/leadership will always be on the look out for new risks. It will then ensure that those risks are managed in the most appropriate manner (including monitoring them but not acting). In a risk managed culture, you make sure that the needs of the different groups are balanced according to the context.

As an IT Risk Manager you know that the most significant source of risk arising out of your teams come from the individuals, relationships and groups. As an IT Risk Manager you understand that the best way to manage those risks is ensure the individuals, relationships and groups under your care are as healthy as possible. It is important to know the people you work with well enough to know that their needs are met, otherwise there is a risk that they will be become ill, or their relationships will suffer. You have to care about your colleagues. Personally, I find the best way to care is to first get people out of the office, and then share with them as they share with me. In other words, put them in an environment where they feel human and then treat them like a human rather than a colleague, or an employee, or a boss, or a subordinate. For this, I am eternally grateful for the numerous coffee and tea houses around London.As for my needs… I write this blog to share my experiences and ideas. My needs are met when people show appreciation, or challenge me with feedback that is useful to help me learn and evolve my thinking so that I can better meet my own and other’s needs. My needs are not met when I receive negative feedback that provides no guidance or insight to help me improve. I consider those that judge harshly or make cruel comments without providing useful feedback to be Internet Trolls. After a pattern of behaviour I tend to ignore those individuals. Does this post meet your needs?


Changing Culture

Seeing Culture touched on the phenomenological* aspects of Culture. This post offers a hypothesis on an epistemological* approach to changing culture. For now, I intend to ignore the ontological aspects, or rather “What Culture really is”.

Sense maker is a tool for understanding culture, and identifying weak signals. When it comes to culture there are weak signals, but they are often strong signals according to other cultures as well. This post is based on Dave Snowden’s idea that the way we change culture is create more cultural agents (stories) like “this” and less like that. So how do we create stories like the ones we want? When we talk about cultural change we mean moving the culture in a particular direction. It might mean moving a traditional (or risk averse) culture to an innovation (or risk managed) culture**. I hypothesise that the cultures of organisations naturally tend to move towards risk averse as they scale and the challenge is to move towards risk managed (innovation). We could use Sensemaker to find the stories but I propose a hypothesis that we can create stories based on our ability to see the different cultures. This is a complimentary approach to Sensemaker rather than a competitor. As the opportunity presents itself I will test this hypothesis and it would be great if you could as well (and feed back your results). It will form part of the Cukeup session I will be running in March.

The approach is quite simple.

1. Help managers / leaders to see culture using the behaviours in Seeing Culture. Get the managers to practice and reflect on seeing culture.

2. Once the managers / leaders can see culture, get them to agree on how to address each behaviour. In Cynefin terms, where to add or remove energy. This is to ensure consistency. e.g. When to praise someone privately / publicly. When to rebuke someone privately / publicly. When to coach someone privately / publicly etc.

3. Get the managers / leaders to mine and share the right kind of stories. The difference between this approach and Sensemaker is that we generate lots of options and let the fitness landscape choose which stories propagate. Sense maker is more precise approach to identify the stories with the most agency and impact. The difference is that between a shot gun and a sniper’s rifle… complimentary rather than competing. And to be effective, both should be used. ( I am keen for feedback from experts on Cynefin on this point )

4. Use Sensemaker to scan the culture for impact.

This is a hypothesis I will be testing. Please join me and share results.

More importantly, please tell me where this hypothesis is flawed. I would prefer to fail it early so that I can find and develop another hypothesis.

* – For Dave Snowden’s benefit so that he can see I have actually been listening.

** – People take risks in a risk averse culture. Its called gambling. Individuals and groups in a risk averse culture have an aversion to risk and tries to ignore it or transfers it others.


Follow

Get every new post delivered to your Inbox.

Join 77 other followers