Are We There Yet? Measuring What Matters

Are We There Yet? Measuring What Matters

Takeaways

“Every time you have a success and pass a milestone along the way of the change process, stop and celebrate so people can see that it’s worth it.” Jack Welch

Are We There Yet? Measuring What Matters

What it Means

Executing against your roadmap is the best way to move from the status quo to your desired vision. Being able to measure progress provides you with an objective way of assessing the initiative’s speed and how you fare against the planned milestones.

Why it Matters

·         A strong measurement system provides visibility around your initiative’s progress

·         It creates an objective and universal way of keeping score, understandable by everyone in the organization

·         Provides an opportunity to recognize when a milestone is reached and to celebrate successes

·         A good measurement system give you an early warning if the rate of progress is slowing down

·         Coupled with your Vision statement, it reinforces your message and maintains focus on the initiative

“Talk about the vision for change over and over and over again.” Jack Welch

Action Plan

Are We There Yet? Measuring What Matters

Your initiative is underway and you need to make sure it stays on track. The best measurement systems help you keep an eye on what really counts and provide you with an early sign when progress stalls. The activities that follow will:

·         Present examples and ideas on how to drive metrics from your Problem or Vision Statement

·         Provide you with examples of typical metrics you can use for your initiative

·         Identify the typical components of a one-page dashboard

Your Starting Point

1.       When I review the original Problem Statement and the Vision, what are the key items I want to change?

2.       For each of these, what would be a good measurement system? There should be five key characteristics of a good measurement system.

3.       Does my organization already have measurement system in place needed to track the items that will change? If so, how well do these align to the five key metrics identified above?

4.       If I don’t already have a measurement system in place, who can help me create one that gets at the data I need?

5.       Is there already a one-page summary (or dashboard) that the Executive team likes to use? If so, I can use this to measure the progress of this project and still track the items I need to track?

6.       Are there any other internal best practices I should adopt for reporting progress?

Quick Wins

The following activity can immediately provide you with a set of measurements for your initiative. Remember, the goal of a measurement system is to create a simple, straightforward way of tracking progress. You may be tracking several items at once, buy your summary needs to be self-explanatory for most people in your organization.

Getting started on developing a measurement system doesn’t have to be complicated. To take the first steps, you need to address three key items:

1.       Look at your Problem Statement and Vision. What are the key things your initiative is trying to change? Remember, just focus on the major ones so that you can narrow your list to the top three to five items.

2.       For each of these items, ask yourself what would be the best way (i.e., the metric) to measure progress. As an example, if you are working on improving sales, it could be total revenue or it could be number of contracts, etc. If you are trying to improve customer satisfaction, it could be a satisfaction score on a customer survey.

3.       Finally, identify which of these metrics already exist in your organization. Chances are, most of the metrics exist today and some may even be included in one of the organization’s data systems. If the metric does not exist today, don’t worry, I will provide more suggestions.

Note:

Once you have brainstormed this list, you will probably already have a few foundational metrics for your measurement system. Most likely, you will also have some gaps you will need to close. For example, for some metrics you may not have historical data available or a good measurement system in place. Team up with key individuals in your organization who know how to extract that data from your systems or create new measurement processes.

Building your Key Metrics Plan

You cannot improve what you cannot measure—implementing an adequate measurement system is critical for your initiative’s success. Most organizations already have abundant measurements in place, so the key is to identify the few metrics that matter for your initiative.

“Measure what can be measured, and make measurable what cannot be measured.” Galileo Galilei, 16^th century Italian physicist, astronomer & mathematician

1.       Collect the Problem Statement you had created with your team and your summary Vision

2.       Carefully read each one and write down the key items that will need to change in your organization

3.       For each item, determine what would be the right metric to gauge performance. Remember the five requisites. A good measurement:

a.       Is fact based

b.       Is easy to understand

c.       Measures what matters

d.       Provides specific examples

e.       Holds people accountable

4.       You should be looking at a list of 3 to 10 different metrics. Your goals will be to reduce these down to 5 or fewer critical ones. Beware! Don’t fall victim to the mistake that many organizations make of getting trapped in the “measurement game.” You can track hundreds of different metrics, but the key is to maintain focus on the vital few, helping people maintain focus and establish new behaviors.

Imagine you are a part of a company whose Vision is to become more customer oriented and improve the overall perception of service, and that you are facing the following Problem Statement:

            “Our organization has an issue with customer satisfaction and this is hurting our sales. We need to be more focused on our customer if we want our company to survive. From some initial analysis, we identified that there are two key reasons for customer dissatisfaction: 1) We sometimes ship the wrong color product and 2) Our organization is not very good art handling customer returns, resulting in delays for replacement shipments to the customer and sometimes missing refunds.”

Below are the main components of the statement and some key metrics we could track:

·         “Issue with customer satisfaction” = Customer Satisfaction survey

·         “hurting our sales” = Sales revenue

·         “customer dissatisfaction” = Customer Satisfaction survey or number of complaints

·         “we ship the wrong color” = Number of wrong orders

·         “delays in replacement shipments” = Cycle time to send a replacement; cost of extra shipments

·         “missing refunds” = Number of returned products that were not refunded; $ refunded and therefore not recognized sales

From this high-level Problem Statement, we could develop several different metrics. The question is: which metrics truly matter and how do we keep this list as short as possible? To guide your answer to that question consider the following suggestions.

Determine whether your measurement system is directly correlated to your desired behavior change.

In our example, we know we need to focus on “customer satisfaction” to benefit the company and that this lack of focus is hurting our sales. Would sales revenue be a good metric for our initiative? Probably not. Sales can be driven by many other factors, which are completely independent from our initiative: there may be seasonality in the sales pattern, a competitor’s special promotion may hurt our sales, our customers may be stocking up and increasing our sales temporarily. Be mindful that your metric must measure, as loosely as possible, the behavior you want to drive. It would be unfair to reprimand or celebrate with your organization for a change in the metric, when your initiative did not drive that variance. In our example, a better way is to measure the identified key drivers of customer dissatisfaction.

Create measurement systems if they are not available.

Your organization may not have a measurement system in place to collect the data you need for your metric. In our example, we would need to find out how the organization measures customer satisfaction today and whether that measurement is adequate to be a metric for the initiative. Ask yourself the following questions:

·         Is the measurement system clear?

Does is have clear guidelines and is the data representative over the long time period?

·         Does it provide enough data?

If your project is scheduled to last 9 months, you cannot live with a once-a-year customer satisfaction survey. And, even if the request for feedback went out after each customer purchase, do you have enough data points to discern a change in customer perception? Do you have 3 sales per month or 3,000? How many data points would you want to see in order to comfortably say the customer experience has improved?

·         What can you do if the current system is insufficient?

If the measurement system does not exist or it’s weak (e.g., information is circumstantial and not objective), how can the organization create a measurement system to set a baseline and track progress?

Leading vs. Lagging Indicators

It is generally better to focus on metrics that will predict your future performance (aka leading) versus those that provide you a snapshot of past performance (aka lagging). This is an important distinction for a few reasons:

·         Lagging indicators always present you with historical data

Since the goal is to change behavior, the best data provides you near real-time representation of performance. Imagine the team working at a fast-food restaurant and ensuring they minimize the wait time for customers. Their visual cue is the number of customers waiting to be served. The longer the line of people, the clearer they need to expedite service to shorten wait time. Now, imagine they received at the end of the week a report with the amount of time people waited in line. This would be helpful, but would not be an immediate performance gauge that would help them serve customers at the present time.

·         Lagging indicators may be prone to indirect drivers

Many organizations use financial metrics from the accounting department to measure progress. The good news is that you generally have consistency in the results. The bas is that the results generally are only available at months’ end and that there may be some accounting considerations that may skew the results artificially. For example, if you measured your progress in Sales revenue and the company sold products globally in different currencies, your sales amount may be artificially skewed by changes in the foreign exchange rates. This could be a driver completely independent of your project.

·         Lagging indicators can help you isolate key metrics that drive specific behaviors

In our previous example, “number of wrong colors shipped,” “cycle time to ship replacement product,” and “cycle time to issue refund” may be good leading indicators that drive customer satisfaction (lagging indicator).

Measure the cause, not the symptom

One of the typical pitfalls of a measurement system is measuring the symptom (i.e., the visible output) rather than trying to measure the actual cause—which is typically what an initiative is attempting to change. Data is your powerful ally in supporting your metrics but can also be deceiving. A famous and comical example of how data can be misleading was a tongue-in-cheek analysis by a mathematician, Dr. Fischer in Denmark. After World War II, there was a significant increase in childbirths, given the more stable conditions as well as a renewed emphasis in construction. The childbirths drove more babies and increased constructions attracted more storks to nest in the city. Using local data and statistics, Dr. Fischer arrived at the conclusion that “storks bring babies.” When selecting your key metrics, always ask yourself if there could be other drivers affecting your measurement system.

Returning to your example Problem Statement, we would recommend focusing the initiative’s metrics using:

·         Number of wrong colors shipped

·         Cycle time to send a replacement

·         Number of returned products that were not refunded

These are key drivers that, if addressed, should ultimately benefit longer term sales and customer satisfaction. As a secondary measure, you would also include these latter metrics, but they would not be your immediate measure to determine if your initiative is on track.

As part of this initiative, your next step would be to set your baseline (i.e., your starting point/status quo) and then your goal (what your Vision defines as “success”). Also remember to include individual milestones between these two data points; you will use them as part of your roadmap and to celebrate successes as you reach each of them.

Game Pages

Your Starting Point

Template to Derive Key Metrics

 

                                                                                                                             

Key Metrics and Definition of Measurement

Metric	Definition

Qualities of a good measurement system

1.      Fact-based

             Good measurements aren’t based on biased criteria and the data needs to be reliable. You need to include all data points, both the positive and the negative, whether it’s financial information, staffing changes or on-time deliveries. Also, the way the measurement is taken should be consistent throughout the project and after the change is in place. Ensure that you always capture the starting point and set a clear target to define success.

2.      Easy to understand

             Everyone involved in the change process should be able to track whether the initiative is making a difference. As an example, many companies use numbers-based customer satisfaction survey or percentage on time deliveries to measure their baseline and set specific targets.

3.      It Matters

             Measure what matters to your customers and your organization. How does the change impact your customer, your productivity, our organization’s compliance to a policy, or whatever it is that matters most with the change that you’ve made? Show why your measurement matters and how improving that metric ultimately improves the organization’s performance.

4.      Provides specific examples

             Highlight your early victories and show how things are better as the changes are implemented. Use a baseline from where you started to show if you’re making measurable progress. If it makes sense and the data is publicly available, also include a comparison to your competitors. This drives a competitive spirit and keeps the organization on the positive track.

5.      Holds people accountable

             Make sure that each of your metrics have owners, especially for internal metrics that are key to success. Visible metrics are a great way to drive accountability and performance. When a leader’s name is associated with a metric, they’ll typically work harder to stay ahead of the pack, or at least to avoid being on the bottom of the chart. Many major organizations use progress reports that include traffic light symbols (Red, Yellow, Green) to quickly provide a visual indicator for the state of each of the businesses. Remember to make it fair: if you are evaluating someone’s performance, make sure the metric is objective and consistent.

My change initiative metrics

Use the following table as a checklist for the metrics you will be using to evaluate the progress of your change initiative.

What am I measuring?

Characteristic	Metric	Metric	Metric	Metric
Fact-Based Yes/No?
Easy to Understand Yes/No?
It Matters Yes/No?
Provides Specific Examples Yes/No?
Holds People Accountable Yes/No?

Appendix

Each organization generally has a dashboard template. If you need to create your own dashboard, at a minimum, you should include:

Initiative’s name

Project sponsor

Project owner

Dashboard date

Executive summary (Remember to mention key milestones achieved and other progress made. List any key issues or roadblocks that are being tackled).

List of key metrics (this could be a chart)

·         Current performance

·         Target performance

·         Last data point (to show progress)

Next steps / next milestones

Current issues and remediation steps (optional)

Key risks that are being managed (optional)

Dashboard date