By Proteus Duxbury and Zaid Masud

Electronic dashboards and scorecards are powerful enterprise tools that provide executives with quick insight into business performance. They can be custom-built or based on reporting solutions offered by a number of vendors. Many organizations have come to rely on the key performance indicators (KPIs) found in dashboards. A brief analysis of KPIs often highlights important trends that can significantly impact strategic performance improvement initiatives. Dashboards have matured significantly over the last decade and have evolved into rich solutions possessing comprehensive graphical and tabular reporting capabilities.

However easy they may sound on paper, dashboard implementation projects can be extremely challenging because of complexities that aren't obvious to the inexperienced dashboard development team. Experience shows that such complexities, if not mitigated early in the project, can cause unnecessary delays or even project failure.

It's our experience that a dashboard project needs to be treated like any other enterprise IT initiative and attacked with rigor and diligence. In this article, we discuss some common mistakes in dashboard implementations. If you can avoid these pitfalls, you'll significantly improve your chances of a successful project.

1. Defining too many KPIs. It's all too easy to get carried away during your KPI design process. Defining a large number of indicators is a common mistake, and it takes time, patience, and peer reviews to ensure that your KPIs are consolidated to the smallest possible MECE (mutually exclusive, collectively exhaustive) set. You might find it useful to start with a longer list and work it down to a smaller one.

Bear in mind that the complexity of your dashboard and the effort required to design and implement it will increase exponentially as the number of KPIs increases. KPI design is complex, and gaining consensus on how a KPI will work is time-consuming. If you have lots of KPIs, you'll probably need to extract the data from a large number of data sources. As a result, development will be slower and user acceptance may take longer because the dashboard will likely become unwieldy and hard to understand.

Here's how to reduce your KPI set:

• Extract dimensions. If you have three KPIs for three products, for example, consider consolidating these into one KPI that features "Product" as a dimension.

• Normalize your KPIs. You may find that your initial list contains KPIs that are not mutually exclusive. Two KPIs may measure similar aspects of performance in different ways or contribute to more than one aggregated metric. One effective technique is to make a list of all source data fields required in your KPI set, then count the number of KPIs in which each source data field is used. Examine all KPIs that share the same source data field to see if there's room for normalization.

• Take your time. Give your KPI design process the time it deserves. Trying to complete it in a day or two may generate a lot of output, but it won't produce the best results. Consider putting the KPI set away for a while and reviewing it later with a fresh eye.

As you work through this process, make sure that your KPIs are MECE and well focused, and that ownership of the indicators is shared across the business rather than limited to specific segments. Also, ensure that your design allows for occasional redefinition of KPIs as target areas of the business improve and new challenges come into focus.

2. Implementing your solution before fully considering the structure and behavior of your metrics. Since the desired behavior and structure of a KPI can be complex, an initial attempt to fully design each one should be made before selecting the technology and implementing the KPIs. If you choose your reporting software before considering the KPI requirements, you may wind up getting stuck with a solution that won't support your metrics as effectively as an alternative might.

At the start of the project, you should consider the structure and behavior of your KPIs in reasonable detail. A KPI's structure includes its text; its hierarchy (i.e., its subsets, if any); and its dimensions. A KPI's behavior includes considerations such as how it's banded, how its value is calculated from the source data, and how that value behaves when aggregated.

A good approach is to record the design of your metrics in a spreadsheet and then create prototypes that can be reviewed collaboratively to ensure that you start with the end in mind. Experience shows that these reviews will be iterative in nature, so don't be surprised if it takes time to gain consensus from key stakeholders. To further validate your design (and as a sanity check), consider manually calculating your suggested KPIs using historic data.

You should consider these eight key factors to ensure a comprehensive design:

• Hierarchy. KPIs can have "parent-child" relationships, so consider whether a higher-level KPI can be modularized into smaller but still useful KPIs that, when combined, calculate the parent. Be careful, however, to keep hierarchies simple. A good rule of thumb is that you should carefully reconsider a three-level hierarchy, and any more levels than that is probably too much.

• Aggregation. Child metrics can be aggregated into parent metrics, and parent metrics can be exploded into underlying child metrics. When calculating the value of an aggregation, you can use simple addition or averages, or you can use a more complex calculation such as a weighted average, median, or percentile range. Sometimes, complex multidimensional expression (MDX) formulas can be used to define aggregation logic, and these can be articulated independently of the selected technology (most tools support MDX). Double-check that all aggregations are accurate because the obvious aggregation may not always be the correct one. Bear in mind that dimensions can also be used to aggregate data, as we explain below.

• Dimensions. Across what dimensions will you want to aggregate and filter your data? Time is an obvious choice, but also consider other useful dimensions within your specific business context. Dimensions will have parent-child relationships with other dimensions in the case of a "snowflake" schema design -- for example, a geography dimension might consist of countries that are split into states, which are further split into counties. When you aggregate using a parent dimension and the data is summarized, you need to consider the rules for calculating summary values. For example, you need to know the rules for what happens to the data if you summarize from daily to monthly.

• Banding. Be sure to clearly define your banding rules at each level of aggregation. Green, amber, and red constitute the most common and effective banding structure. Care is needed when defining how the banding will behave. How do you know when the metric moves from green to amber, for example? Will the banding boundaries be spread equally across the possible range of values, or will some other scheme -- for example, absolute values or an MDX formula -- be used to define the behavior?

• Weighting. A parent KPI may be a weighted summary of its child KPIs; however, these weightings will need to be carefully considered to ensure that the summarized KPI accurately reflects its definition.

• Direction. As obvious as this may sound, you need to make sure that it's clear whether a high value or low value indicates a positive result. For new people unfamiliar with the KPI, this can be a source of confusion. Also, ensure that scales and measurements are clear. For example, does 0.1 actually mean 10 percent?

• Ownership. Assign a business owner for the design of each KPI (one owner may own multiple KPIs). This ensures that someone is monitoring its utility and can suggest changes as the KPI comes into common use.

• Data source. Ensure that your organization has data source(s) that contain the information needed to calculate the KPI. If you discover that electronic source data is not readily available, the scope of the project will increase; this is an issue that needs to be raised sooner rather than later. Manual entry of metric data can be time-consuming and complex, and if you need to support it, make sure that you have a tool that can easily be deployed for this purpose.