Author Archives: theitriskmanager

About theitriskmanager

A IT programme manager specialising in delivering trading and risk management systems in Investment Banks. I achieve this by focusing on risk rather than cost. A focus on costs can lead to increased costs.

Exapting Sensemaker to Agile – An Experience Report.

I have been a fan of cognitive-edge technology and techniques for many years. Myself and a small group have been engaged in exapting sensemaker for use on Agile Projects. I did my Sensemaker training with Tony Quinlan. Tony subsequently delivered a fantastic introduction to Sensemaker at QCon which is the material I use when introducing it.

Sensemaker is a fantastic and valuable tool, though its usage in Agile transformations and projects is different to its original context. I will share my experience using the standard report format namely:

  1. What is the context?
  2. What did you try?
  3. What worked?
  4. What did not work?
  5. What have you learned?

Context for using Sensemaker

In all three cases, there was a situation with a lot of uncertainty and the leaders showed great strength by trying something new.

  1. In the first example, a leader heard about sensemaker and very bravely decided to use it to get unstructured 360 feedback from a large group of people.
  2. In the second example, the team had scored “poorly” on one question on a company wide survey and the leaders wanted to understand why.
  3. In the third example, the leader for an Agile transformation (two thousand people) wanted a survey to understand how people in the department felt about the transformation. A pilot with fifty people was run to test the process.

It was only because I had run the first two fairly small sensemaker exercises that I had the confidence to run the much larger (and personally risky) third exercise.

What we did

In all three cases, the leader sent an e:mail out setting the context for the survey and asking for the team to participate. The sensemaker survey they were asked to fill in was a slide in a powerpoint presentation. In the first example, the survey was also printed and handed out in a team town hall. In each case, the leader introduced the sensemaker survey in the townhall or extended leadership meetings.

The responses were all anonymous. In some locations a “post box” was created for the team to post their response. In some locations, team members would send their response to a trusted colleague who would forward them to the person collating the results. Many people felt comfortable enough just to send a response directly.

Unlike the sensemaker method which starts the signifier creation process with a literature search, the triad signifiers were created using terms commonly used in the environment. For the 360 degree feedback, the leader incorporate the six headings in the corporate leadership assessment. For the Agile transformation, phrases common to agile and software development were used (Technical Debt, Architecture, Deadline, Business, Leadership, Culture, Mastery, Autonomy, Purpose, Business Value).

The results were collated manually. Each dot on a completed survey was collated onto a large triad, one per powerpoint page.

Slide1

The sentiment (positive, negative, neutral) was depicted using colour (red, grey, blue respectively) and strength of sentiment ( normal, very) by shape (dot, circle). Red / Blue was chosen as traditional red/amber/green are difficult to differentiate for those with colour blindness which is common in technology.

The results were summarised. The summary and original survey sheets (or scans of physical paper) were included as an appendix to the summary. The individual stories were ordered by sentiment, and a hyperlink was placed on each summary diagram to the first story of each sentiment to speed up finding the stories.

The results were played back to the leadership team and then to the participants and the wider group. Each was invited to interpret the summary triads for themselves for before discussing them, and to read the stories associated with each cluster.

What worked

Sensemaker works! It provides useful and actionable insights.

One of the key experiences is that sensemaker shows you what people are not telling stories about. The gaps are as telling as the positive or negative clusters. For the first example, the leader was a very accomplished practitioner in the technology, however their other abilities outshone their technical skills. Armed with this knowledge, they were able to “create more stories about their technical abilities”.

One of the most powerful aspects of Sensemaker were the conversations that the exercise prompted each time. It provided a focus for some very difficult conversations that were based on data points (stories) rather than opinion. It enabled the group to make sense out of situations. Many of the things that Sensemaker revealed were already known, however they were difficult to discuss as opinions without “Hippos” closing down the discussion. Getting data to support a hypothesis with a traditional likert scale survey would have been politically unacceptable however the Sensemaker reveals the data without conflict. Once revealed, the stories can be discussed.

When you do a Sensemaker exercise you realise how beautifully it fits with the Cynefin framework. Discussions easily reveal things that are obvious or have multiple hypotheses. Sometimes, the discussions reveal it is necessary to dig deeper or bring in an expert. There were also stories that simply require action without understanding. Sensemaker helped us shape the transformation and resulted in several specific changes.

The three exercises only involved a few dozen stories rather than the massive number of stories that Dave Snowden and Tony Quinlan describe. Even though they numbered a few stories, they still provided valuable insights. Sensemaker works on the small scale as well as the large scale. Within organisations, culture tends to be determined at the tribe level (dunbar number of one hundred and fifty or less) with a dominant individual or group of individuals defining the culture.

Starting small helped me learn about Sensemaker. The first sensemaker exercise was small and fairly low risk. The experience at this level gave me the confidence and case study to refer to for the riskier second exercise. My experience with the first two exercises exercises helped me better understand the issues and provided me with stories that enabled me to convince the leaders for the third exercise. Starting small was safer for the leadership team and let them experience the approach before they tried it on a wider group outside of their area of control. My main practical advice would be start small and grow.

The leadership team said they would like rerun the Sensemaker exercise on a regular basis to track the progress of sentiment in the team.

What did not work.

Although the manual exercise was incredibly useful and valuable, reviewing the results was clunky and required a lot of effort. The manual approach is certainly limited in terms of scale. Above one hundred stories, it is likely that it would start to fail, and it would be necessary to use Cognitive Edge’s sensemaker tool. Ideally I would have preferred to use Cognitive Edge’s sensemaker tool but unfortunately that requires spending money. Spending money is the one thing that managers in large organisations do not like to do. The hope is that sharing the experiences with executives will lead to them to freeing up the funding for larger exercises. This is particularly difficult in an environment where people are used to getting their software for free (open source).

Although the results were insightful and valuable, we got stories from significantly less than half the people asked to participate. I worry that we got the “CYNics”(1) who cared but there is a large silent majority who have a different set of stories and opinions. I suspect the cognitive edge software would help with this.

(1) See the Dave Snowden’s references to cynics.

What did we learn

Sensemaker is a great tool to provide insights into your organisation when you are planning or engaged in an Agile transformation.

The normal Sensemaker approach (e.g. Literature search for signifiers) isn’t necessarily needed when the language of the culture is already well understood.

Sensemaker facilitates difficult conversations and is useful even if “no one learns anything from it”.

Even though Sensemaker and Cynefin are valuable when used individually, they are even more powerful when used together.

Next Steps

The Sensemaker and Scaled Agile Practitioner Community need to do more work together to better refine and exapt Sensemaker and Cynefin usage in the Agile Community. A summit focused on the use of Sensemaker in the initiation and monitoring of Agile Transformations would be very valuable. In particular, packaging Sensemaker with Dan Mezick’s Open Space Agility.


Executives and Transparency

One of the cornerstones of scaled agile is transparency. This is particularly true for executive transparency. Unfortunately transparency is a double edged sword and executives involved in transformations normally avoid it like the plague.

water-880462_1920

Transparency allows executive running a transformation to demonstrate that they are a success.

on the flip side…

Transparency shows when transformations are failing.

If you are an executive and you intend to succeed, the first thing you will do is establish transparency. That way, when you are a success, it can be demonstrated rather than a matter of opinion. Furthermore, if you do not have transparency into the transformation, you cannot identify the things that need your focus to improve. As Jabe Bloom says in his 2017 Lean Agile Conference talk, data should provide an invitation to executives to “Go to the Gemba” (Go and see where the work is done). In other words, the data shows you where to go, and then the executive should go there so that they have total transparency into what is going on, not a view of the world filtered through management updates, excel and powerpoint.

The exemplar of this behaviour is Mark Gillett. I worked for Mark helping to build a dashboard that showed the state of the business. Our top three metrics were “Number of customers”, “Activity” and “Revenue”. Mark invested heavily into the accuracy and coverage of these metrics. What he wanted was not available “off the shelf” so he build a world class group of developers who could deliver what he needed. He could drill down from the number of customers into the customers per product, or drill down into “New Customers” and “Churn”. Overall we had about four hundred metrics. Once these core business basics were in place, Mark worked on the operational metrics to give a better understanding of how effective the business invested in the products… lead time, bugs, performance. The key thing is that these were not executive metrics, they were the metrics that everyone in the business looked at. And this is the key point, everyone in the organisation knew that Mark looked at these metrics regularly, and they behaved accordingly. They cared about the results because they knew Mark could see if they needed help.

Jira is a tool developed to help teams manage their development. It is not a tool to manage across teams or at an enterprise level. In order to create transparency for executives, you need an expert who can extract the data you need to create the views they need. One of the graduates working with us created an app to extract data from Jira into an SQL based database. Once the data was in the SQL database it took a couple of days to create an excel report that gave an executive view of lead time using weighted lead time.

Recently a couple of colleagues and I created the first version of a report that allows executives to drill down to a deliverable and then see view the value stream for the deliverable with colour coding for the different teams. It is possible to drill down from the value stream to a cumulative flow diagram for each team. This was all in a couple of weeks part time. The numbers ain’t great but they are better than nothing and they provide the executives with “invitations to the Gemba” as Jabe would call them. (I will shortly be creating an open source product to give executives transparency into lead time and the value stream.)

If executives want to be successful with a transformation, they will demand transparency.

Executive who do not believe they can deliver a transformation avoid creating transparency like the plague. The problem with transparency is that it no longer the responsibility of the team to deliver, but everyone in the organisation, especially executives. Executives who do not “Go to the Gemba” can be held accountable when there is transparency.

So how do executives avoid transparency? Easy, they do not demand it. Everyone in their risk averse organisations will happily avoid looking at the metrics, especially if the executives can prove that they do not care about the metrics.

The benefits of not having transparency include the following:

  • Employees do not need to improve.
  • Employees (especially the executives) do not need to learn new techniques or approaches and can continue using the “same old” methods.
  • Executives can manage using their opinion rather than using metrics to reveal reality.
  • Executives can ignore problems they do not feel comfortable with.
  • No one can be blamed when the transformation fails to generate any improvement.
  • Good people who wanted to achieve results will leave.

So if you want to be part of a successful transformation, look for an executive who is demanding transparency and has strong (but weakly held) opinions on how they will get it. When you go for the interview, ask the executive to tell you when they have “Gone to the Gemba” based on the transparency they have in their metrics dashboard.

If you want a long term coaching engagement with no expectation of delivery of improvement, look for one where the executives do not care about transparency, especially around lead time.

Why I’m going to create an open source lead time tool.

From my experience, if executives do not have transparency into the lead time of deliverables, they do not care about improvement. If they do not care about the improvements, then neither will most of the people that work for them. This makes it hell for Agile Coaches who are bought in to help people adopt new approaches.

Building an executive lead time dashboard requires specialist experience. Without the dashboard, it is hard to create that transparency. The one thing management controls absolutely is spending money which makes it hard to bring in products you have to pay for. Therefore to help my fellow Agile coaches, I intend to create an open source (free) product to visualise lead time and value streams.

More details to follow.


Three Steps to Scaled Agile

Scaling Agile requires the coordination and collaboration of hundreds, thousands or tens of thousands of smart independently minded individuals. To scale Agile, a small set of simple goals are needed that individuals and teams in the organisation can focus on, aware that if they achieve the goal, they are part of something much bigger. They can then use the most appropriate practice to achieve that goal.

letters-3195083_1920

One of the key learnings from our experience at Skype was that we had to keep the process so simple that people did not have an excuse for not following it. If people did not follow the process, it was because they did not want to follow it rather than the process was too complicated and they made a mistake.

To get your head around scaled agile, we need to consider the different “levels” or “concerns” in the organisation:

screen-shot-2016-12-03-at-06-48-13

So what are those things? What are the goals are each level?

Executives

There is one goal at the executive level:

  1. Ensure that the executives have transparency into the system to ensure that the goals are being met.

Portfolio

There are two goals at the portfolio level:

  1. Ensure that the backlog of Epics (single team and cross team) is strictly ordered (i.e. no joint 3rd priority) based on the constraints in the organisation. One of the key constraints at the organisation level is the capacity of individual teams.
  2. Manage capacity by moving capacity to the constraints, and planning capacity for the future. This involves ensuring that there is capacity in the system to meet any commitments that have been made to external parties.

Team of Teams

The goals at the team of team level:

  1. Deliver Value.
  2. Reduce Lead time for the delivery of value.
  3. Reduce Lead time for fixing production incidents.
  4. Reduce the number of production incidents.

The team of teams level is fundamentally about delivering value.

Team

The goal at the team level:

  1. Ensure that delivery is value focused. That means that each Epics (single team and cross team) should deliver value rather than be bucket for stories.
  2. Ensure that team level delivery is predictable and consistent. Consistent in terms of quality. Consistent in terms of size. Predictable in terms of quantity.

The three steps to Scaled Agile

Here are the three steps to scaled agile:

  1. Get teams to deliver value in a predictable, and focus team of teams on reducing lead time.
  2. Create a system of transparency so that you can see how everyone is progressing to achieve their goals.
  3. Bring business decision makers to come together to prioritise the backlog of Epics (single team and team of team).

These simple rules allow the organisation to coordinate and collaborate without having to understand the entire system. From these simple rules, the organisation can generate complex and speedy responses.

The following practices, tools and thinking tools will help you achieve the goals:

AgileOrgStructurev2

So what have I missed? Which goals do the organisation need to achieve in order to scale agile? Comments please.

 


#NoEstimates and SWAG Estimates

Just over a week ago, Vasco Duarte, my good friend and leader in the #NoEstimates tribe posted this tweet during a mammoth twitter conversation about #NoEstimates and SWAG Estimates:

IMG_3908

I realised that there may be some misunderstanding the purpose of SWAGS, and in particular the relationship with the #NoEstimates movement.

lego-3135581_1920

Lets start with story points.

Story Points Versus Story Count

Story Points are crack cocaine for managers who want to believe in a world where they have more accuracy than they really do. Story points, if used at all, should only be used by the development team and the product owner to communicate relative effort. In order to create a story point estimate, the product owner must present a story to the development team that meets the definition of ready for the team. Typically this occurs one or two sprints before the team work on the story. Often the definition of ready includes the product owner taking the story through a “Three Amigoes” session with a developer and tester. This is after the product owner has detailed the story including acceptance criteria, often in the Given-When-Then format. In other words, a large amount of effort is normally required before a story is ready for the team to give a story point estimate.

Mature development teams only work on stories within a defined size range and drop the need for story points altogether. Given the number of stories to be developed, an estimate of the elapsed time to deliver an Epic can be just as easily be determined from the story count as it can using story points. The advantage of using the story count instead of story points is that an estimate of the duration can be made before any work has been done on the story at all.

So within the sprint, some teams choose to use story points and often more mature teams will simply rely on the story count. Outside the sprint, story count is the only sensible option for estimating the lead time for an Epic that spans more than one sprint.

Whence SWAGs

Story counts are useful for Epics when you know how many stories need to be developed. However, there are times when we do not know how many stories are needed, and often when we do not even know what Epics are involved.

This is where SWAGs or Sweet Wild Assed Guesses are useful. (At Skype, that is the name that we put on the field label in Jira).

Capacity Planning

At Skype, a number of us including Tony Grout and Lisa Long developed Capacity Planning, a mechanism to coordinate all of the development of Skype and Lync (now Skype for Business). The Product Executives and Managers would prioritise the backlog for the next quarter based on the capacity of individual teams. A large number of the initiatives that went into capacity planning did not make the cut because they relied on work by a team whose capacity was being used on higher priority initiatives. As a result, we needed a cheap and simple way to estimate the effort required from a team. We used a SWAG, an estimate of how many weeks it would take a team to develop a piece of work. The SWAG was good enough to help us understand how much work a team could do in the quarter. It was understood that SWAGs were not accurate and we frequently referenced the work of Todd Little and Troy Magennis that stated the actual versus estimate followed a log-normal (Weibel) distribution with a most likely actual being double the estimate.

The SWAG estimate was used in the Capacity Planning Process that occurred weeks and months before the stories in the epic were broken down into stories. Once the stories were available, they were used on some initiatives to track progress using just the count of stories.

Subsequently Dan North introduced me to his client, a large American bank, were I introduced them to the same process. Since then the approach has been used in further organisations, large and smaller.

The key issue that Vasco’s tweet asked if we had compared estimates with story counts… The problem is that the story count does not become available until weeks and months after the capacity planning session where we needed them. Not only that, but we did not want to put the effort into breaking down Epics into stories when we might not even build the Epics if they were blocked by other work.

As a result, I suspect that SWAGs and #NoEstimates are complimentary rather than in conflict with each other.

 


I was wrong about Culture – “Pasted Values”

I’d rather be wrong than right. For some time it appears that my thinking about culture has been wrong. It probably explains why I have had little success trying to change it. I still think that seeing culture is an important first step. However, the model about the underlying principles that drive culture is obviously wrong. (The model is presented here and here).

The model is based on the work of Edgar Schein (My mentor Marc Burgauer introduced me to his work). Edgar Schein’s model talks about espoused values and assumptions. I had interpreted this to mean that behaviour in an organisation (in a Karl Weick sense) is driven by the underlying value function of the organisation. To represent the value function, I use the financial definition as it is the most useful and general model that I’m aware of and can be used to model learning. The hypothesis was that if we can change an organisation’s values, the behaviour would change.

FeatureInjectionValueFunction

Today whilst listening to Charles Duhig’s “The Power of Habit” I realised that my hypothesis was wrong and I had been ignoring the obvious. Culture is not just an expression of an organisation’s values, it is also based on a set of organisational habits.

Imagine an excel spreadsheet where we write some function:

=if(X = Y,”do A”, “do B”).

If X = 10 and Y = 10, we would get the result “do A”. If X = 10 and Y = 11, we would get the result “do B”. Currently X = 10 and Y = 10 which means the function returns “do A”.

Now if we change the excel function to be:

=if(X = 2Y,”do A”, “do B”).

If X = 9 and Y = 9, we would now get the result “do B”.

That was my naive interpretation of culture. All I needed to do was help people see a new way of looking at the world (the phenomenology), a new value function and the behaviour would change.

What I had failed to understand was that most people do not re-evaluate their value function every time they do something. Instead, its as if they copy the Excel Cell and “Paste Value” so that the sell always contains the value “do A”. Either that, or they have turned off automatic calculations and need to press <F9> to update the value in the cell to “do B”.

In “Thinking Fast & Slow”, Daniel Kahneman talks about System One and System Two. System One is automatic whereas System Two is the one that does critical thinking and evaluation. We need system one which is fast and unthinking because system two is slow and takes more effort. In effect, when we encounter a culture, most of it operates in System One and my solution was to engage System Two.

So I’ve broken my model of understanding for culture. I was wrong. My prize is that to understand culture change, I need to know about “habit change”, addiction, CBT, and a bunch of other stuff . If you know of any good resources or other things I need to know, please leave them in a comment.

Today I realised I was wrong. Now I have better chance of a better tomorrow.


BDD Done Easy – The three types of data

There are three types of data that can appear on the output of a system. The three types of data are determined by the mechanism that is used to provide the data item.

The three types of data are:

  1. Data entered into the system from outside the system
  2. Data calculated in the system with data in the system
  3. Data created by processes in the system

machine-3039352_1920

To describe the three different data types, consider the following example report. Students are allowed to withdraw money from an account. A limit is applied to how much a student can withdraw each month. A student can choose to override the limit but any override needs to be approved by a responsible adult. This report shows those transactions that override the limit.

  • In order to identify whether appropriate limit breaches are being approved
  • As a Parent
  • I want to see transactions that breach the limit, along with the reason and the approver.

Screen Shot 2018-03-25 at 18.39.27

Data entered into the system from outside the system

Screen Shot 2018-03-25 at 18.39.51

In this example, the Student Name, Monthly Limit, Withdrawal Amount and reason to override the limit. All of these data items need to be enter either via a user interface or a system interface.

Values are either entered free format, or selected from a list of permitted values (The items in the list of permitted values are originally entered free format.

Data calculated in the system with data in the system

Screen Shot 2018-03-25 at 18.40.06

These are the values that are calculated within the system. In this example the balance and the % above limit. To indicate that these are calculated value, I prefix the name with “get”. Account.getBalance and Transaction.get%AboveLimit.

Account.getBalance(QueryDate) = Sum(Transaction.Amount) where transaction.date before QueryDate and after 1st Day of monthMonth

Transaction.Transaction.get%AboveLimit = ( Account.getBalance(Transaction.Date) – Account.Limit ) / Account.Limit )

Data created by processes in the system

Screen Shot 2018-03-25 at 18.40.23

These values are created by processes in the system. In this example, Date, Approved by and Approval Date. These values are all populated in a THEN in a GIVEN-WHEN-THEN statement. We start at the last step in the process which in this case is the approval.

  • THEN Approved by is set to the logged in user.
  • AND the Approval Date is set to the current Date & Time.
  • WHEN the withdrawal is approved
  • GIVEN the approval screen is displayed
  • AND a withdrawal above the limit is on the screen
  • AND the user is logged on
  • AND the user is an approver for the Account

Similarly (and ignoring the approval e:mail steps in between)

  • THEN the Date is set to the current Date & Time
  • AND an approval is sent to list of users who are approvers for the account.
  • WHEN the student overrides the limit
  • GIVEN the student is on the limit override screen

Understanding the three types of data means you ensure the correct approach for populating them is adopted.

This blog post was inpired by a conversation with Rekha Kusumanchi about the Cotswold Way. My thanks to Rekha.


BDD done easy – The Cotswold Way

The Cotswold Way is an Agile approach to business analysis, developed by Kent McDonald and myself.

IMG_2048

Before you start, you need to identify whether you should be doing Business Analysis, Product Management or JFDI. Cynefin is a particularly useful tool to determine which approach.

The problem with BAs and Story writing in general is that most people think of it as an art form or a brainstorming activity. In fact there is a very simple directed process that Jenny Martin elegantly summarise as OOPSI (Outcome, Output, Process, Scenario, Input), and if you do it backwards, it ISPOO…

The process that I now refer to as the Cotswold Way is a further elaboration of the approach that Kent and I created called Feature Injection.

There are seven simple steps in the Cotswold Way:

  1. Hunt the value. Start where ever people describe the problem/solution. Then work to the output and get them to sketch the reports that they need. They should be able to explain how and why they need the  output (report). This is the customer value which is not the same as the business value.
  2. Once you have the output, work backwards using the information smells process to identify the data and calculations you need to produce the report. This makes creating examples super easy. (For those who prefer videos to articles).
  3. Now build the output in a spreadsheet using the model developed in step 2.
  4. You can then work backwards to describe the behaviour of the system using the Given-When-Then format to define the process that populates the required data. (See here and then here )
  5. Depending on the size and complexity of your organisation, you need to meet with either some lead developers/QAs or Architects/Lead Devs/Lead QAs to agree the architecture for the solution. Once you have done that, you can break down the solution into Epics and assign stories to the various components etc.
  6. To break the solution into Epics, you now work backwards for each system to define the Given-When-Then. (Obviously if you are doing everything in a single component, you can ignore steps 5 & 6.)
  7. Once you have your examples and Given-When-Then at the system levels, its a simple matter to slice them up into stories. David Evan’s article, and book are great for those who want to improve the stories they write.

This process takes HOURS rather than days.

Want to get started? Its simple, ask your user or business stakeholder to sketch the output they want rather than describe what they want to you.

Blog posts providing detailed examples of the steps above to follow.