Pitfalls in setting KPIs

In my last post, I wrote about the two types of performance indicators we can define and track. These indicators or KPIs, when defined well, have several benefits. When we write down measurable goals, we are 33% more likely to achieve them; KPIs help us stay accountable to these goals. They are also an effective means of making sure that the entire team is pulling in the same direction.

Setting realistic and relevant KPIs is not easy and there are many pitfalls.

  • What we measure does not contribute to the organization’s goals
    KPIs are useful only when they contribute to the business. For example, leaders wishing to build a Center of Excellence should define KPIs for their CoE managers that address IP creation, publication of white papers, prototype creation, customer ratings on innovation, process improvement, revenue growth and reliability metrics. If your managers’ KPIs have not been updated to reflect your organization’s priorities, their work will not contribute to its strategic objectives.
  • What we measure has unintended consequences
    In 2019, Boeing targeted the reduction of training costs related to the 737 MAX. This led to pilots getting a short training on the new features of the aircraft on an iPad instead of a flight simulator. The pilots did not learn of a new system that would force the aircraft to dive under certain circumstances and how to overrule it. This led to two catastrophic crashes.
    Some companies track the number of training hours completed by every employee. With no importance given to the relevance of the training or its effectiveness, this quickly becomes a “tick in the box”. In one company, under the threat of losing a high-performance rating if they did not complete 30 hours of training by the end of the year, some senior managers rushed to complete online courses on meeting and email etiquette.
  • Focus on only financial targets
    While the company needs to be profitable, a preponderance of financial KPIs can confuse and demotivate non-executive staff. Can you create a separate Financial KPI Dashboard for senior leaders and executives?
  • Relevance and “the big picture” are not established
    In companies that have a strong culture of performance management, the goals and objectives of the business are defined early in the year and cascaded progressively from the leadership level to all levels of the staff. Particular attention is paid to how relevant and inspiring the goals and objectives are. A “best practice” here is to involve the team when defining their KPIs.
  • The data collection process is biased
    Years ago, a tech company decided to grant annual performance bonuses to their Project Managers on the basis of CSAT scores (these are annual Customer Satisfaction ratings). This data was submitted by the Project Managers themselves; unsurprisingly, none of the ratings was low.
  • Too many KPIs are defined
    The KPIs we set for our employees are a powerful guide to what they need to do and how well they are doing it. Too many indicators will cause the really important metrics to drown in a sea of data.
  • Lead Indicators are given short shrift
    Lead indicators are KPIs that help us understand and predict what lies ahead. While Lag Indicators help us understand how we have performed so far, having an over-reliance on these can lead to short-term thinking. Having relevant lead indicators can help us identify how our results are trending and where we can make course corrections. Defining relevant Lead Indicators requires a strategic mindset and a solid understanding of how your business results are achieved.

It is worth remembering that poorly defined KPIs can cause more harm than good. Unrealistic targets can demotivate your team. KPIs that address only the “what” but not the “how” can encourage “winning at all costs” mindsets rather than on behaviours that lead to long-term success. I would love to hear your stories on how you have set KPIs that have made a real difference to you, your team and your organization.

2 thoughts on “Pitfalls in setting KPIs”

  1. Ravi,
    Very true. Here is my experience in running software projects
    1. Very few Projects define KPIs in terms of bug density and Productivity. Most of them define their KPIs by the number of hours and the project risks. Forecasting risk especially the consequences and the time span for which it has an impact on the project especially at the tail end (statistically speaking) is a futile exercise.
    2. Since we measure the number of developer hours on a project, developers are very content in churning out “time-sheets” at the end of the day and this does have unintended consequences especially when we use this data to estimate newer projects.
    3. KPIs do have a Financial Targets, for example, the budget within which the Project should be completed. This is acceptable when things don’t change. But then an appraisal cycle comes up and people on the team members are given a raise. This renders all estimates of budgets meaningless.

    These problems can be avoided if we have metrics that can be measured while work is being carried out and not post facto. Software engineering teams can measure the following
    1. Number of commits in a given time period like a week or month
    2. The Number of bugs discovered during E2E testing
    3. The test coverage

    These are possible if the right tool set is deployed on the Projects.

Comments are closed.