3. Allowing scope creep of KPI requirements. As implementation begins and end-users review progress, they often request changes. However, if they're not carefully managed, changes can impact your dashboard delivery schedule. Your KPI requirements should be treated formally, like any other software requirements.

A Statement of Requirements (SoR) documents the initial requirements and ensures that there's a common understanding on the scope of your dashboard implementation project. It includes MECE KPI requirements and other functional and non-functional requirements.

You should create a formal SoR and manage it carefully throughout your project to avoid scope creep and to ensure that your stakeholders get the dashboard they need. To ensure that value is delivered early to the business, prioritize KPIs to allow earlier iterations to implement the most important indicators.

One approach to prioritizing KPIs is to tag them with one of the following "MoSCoW" [must, should, could, want to] rules:

• Must have -- for KPIs that are fundamental to the system. Without these metrics, the system will be unworkable and useless. The "must haves" define the minimum usable subset, and your project guarantees to satisfy this set.

• Should have -- for important KPIs for which there is a work-around in the short term and which would normally be classed as mandatory in less time-constrained developments. The system will still be useful and usable without them.

• Could have -- for KPIs that can more easily be left out of the iteration under development.

• Want to have but won't have this time around -- for those KPIs that can wait until later. These are key indicators captured in the design process but discarded or defined to be outside of scope.

To minimize scope creep, make sure that your stakeholders fully understand the SoR and that you can achieve their approval and sign-off before commencing development. However, bear in mind that the SoR is not frozen in time; it's a living document that needs to be continually reviewed and updated as the dashboard is developed.

For larger projects, ensure that a change management process is set up that includes a workflow to raise change requests, estimate costs, and proceed with an approval or rejection decision by a project steering committee.

4. Leaving high-risk data sources until the end of the process. The greatest uncertainty in your project before development begins, and therefore the hardest thing to estimate, will be the complexity of the integration points with the data sources that will feed your dashboard. Since these will be disparate systems, they may not readily support the integration that's required for your reporting solution. So the integration points will need to be planned and designed early in the project.

As you design your dashboard development project, plan to tackle high-risk, difficult data sources early. You may find that you have a tendency to implement the easier data sources first to demonstrate early progress; however, if you leave the difficult ones until the end, you greatly increase the chances of falling behind your schedule. Tackling data sources with high architectural risk first will ensure that you have time in the project to account for the inevitable unforeseen issues that will arise.

You may need to deal with multiple input source data: relational databases, text files, Excel, CSV (comma-separated values), unrelational data stores, application exports, and manually entered data and reports. Check that your tool can support the formats that you have so that you won't need to deal with complex format conversions -- don't underestimate the time and eventual maintenance effort that you're signing up for in converting formats. Also, if your dashboard involves batch updates, don't be too ambitious with the frequency. Daily updates are often too aggressive; weekly updates may be more sensible.

5. Overlooking existing technology investments. Many organizations have already invested in dashboard technology without knowing it. Before considering a purchase of new dashboard toolsets, examine existing applications to see what reporting solutions they provide. Other departments or units may own tools that are already providing the functionality you need.

Review your organization's enterprise road map. You may find that certain packages are a better fit with your company's longer-term IT strategy, and this may impact your selection decision.

If your company has enterprise resource planning (ERP) or business intelligence (BI) systems in place, bear in mind that these assets may have reasonable dashboard features. It's worth investigating them to see if there's a fit. You may also want to review your in-house technical skills to determine whether they're sufficient to maintain the dashboard using your selected technology.

A thorough and comprehensive system selection process is critical in ensuring that you choose the tool that will best address your requirements. Take time to diligently perform a full evaluation of the various options. Your specific needs will dictate which application works best for you.

6. Developing all of your data sources and KPIs at once. As with many software projects, it's prudent to develop a dashboard iteratively using best-practice agile principles, including continuous testing, rapid prototyping, time-boxed development, and frequent releases. Because of the complexity of the data sources you're working with and the frequently changing requirements you'll encounter as end-users start to see progress, agile approaches work better than traditional waterfall approaches for dashboard projects.

Aim to implement a data source in each iteration rather than starting all of your KPIs at the same time, which may overwhelm the development team. A divide-and-conquer approach is well suited to the development of your KPIs. Exhibit 1 shows an example of an iterative development plan.

This approach has the following advantages:

• Lower risk. By dividing KPIs into iterations, you improve the project's chances of overall success because at least some KPIs will likely be delivered even in budget-constrained environments.

• Higher quality. Each iteration is completely tested and should in principle be a fully functional release. Early, continuous testing results in fewer product defects.

• Earlier benefit delivery. Since the first iteration implements a complete KPI, users can start taking advantage of the new tool quickly, rather than having to wait until the end of the project.

7. Underestimating testing effort. A dashboard is like any other piece of custom software: Early errors will destroy credibility and user confidence. The structure and behavior of KPIs and scorecards can be very complex, and testing them can be even more complicated. The complexity increases with the number of KPIs in your complete set, and it escalates rapidly if custom code has been developed to calculate and render your facts and dimensions. Develop robust scripts that test every scenario. These should be based on reliable source data sets that reflect the full range of inputs that a production environment generates.

In fact, from a project scheduling perspective, the complex calculations that dashboards typically implement mean that you can expect to spend at least as much time on testing as on development. Relying on users' experiences alone is not enough. You should prepare a comprehensive test plan and staff the project with specialized testers who are experienced in identifying unusual boundary cases.

One helpful technique is to create spreadsheets that manually calculate KPIs and compare the results to your dashboard output.

Key areas that your test scripts should target include the following:

• Dimensions. Ensure that your collection of test scripts has full coverage of all dimensions within each KPI.

• Banding. Check that your banding behaves as intended.

• Hierarchy. If a KPI is dependent on child KPIs, the children must be independently verified, followed by verification of the parent calculation.

• Aggregation. If a KPI involves aggregation (roll-up), ensure that a comprehensively representative sample of results is correct.

• Data source. Compare the dashboard output directly to the source to ensure that it matches, particularly where there are possible data type conversions.

• Time. Confirm that data is being reported correctly over time as a dimension, especially if aggregation is involved. Time-based calculations can be prone to errors and warrant higher levels of testing.