Pseudo Performance Indicators - The outcomes of faulty measurement systems

Measuring performance helps to determine progress towards and proximity to goals. However, flawed indicators from poorly designed or faulty measurement systems may trigger the wrong actions, resulting in poor outcomes. Let’s explore with business, personal development and spiritual examples...

Most systems, including socio-technical systems, often require some parameters, properties, or parts to be at, or interact in, certain proportions for the system to function effectively. Our ability to determine the state of these key items is critical to determining the appropriate actions or tuning that is required to achieve the intended outcomes of the systems. Measurement helps us to understand the state of things, processes, relationships, etc. and is an indicator of progress towards, or divergence from, the achievement of a set goal. From business leaders presenting business performance reports to their shareholders and investors, to individuals determining progress towards development goals, to content developers seeking to gauge the reception of their ideas, or even to production plants where the line between a great day and the next disaster is the quality of the monitoring apparatus on critical systems, the importance of accurate measurements as signals of performance cannot be overemphasized.

Measurement helps us to understand the state of things, processes, relationships, etc. and is an indicator of progress towards or divergence from the achievement of a set goal.

So, what if the measurements are wrong?..

The resulting pseudo-performance indicators from a faulty measurement system drives wrong behaviors that may result in disastrous outcomes for the system being measured with potential impacts on others. It could create a false sense of safety which fuels inaction whereas a quick intervention may have averted an accident or prevent severe outcomes. Examples of these abound, such as seen in many industrial accidents where final investigation reports indicate that certain actions that could have been taken were not taken because key actors were oblivious of the brewing storm. In relationships, some much needed break/retreat that could have reset the course of a crashing marriage, partnership or business relationship is ignored because key players assumed everything was fine driven by the wrong indicators whereas the right perspectives would have shown that things were not doing so well after all.

In addition to giving a fuzzy feeling when you should be taking action, false measurements can also create an alarm that probably sends you down a rabbit hole and distracts from other key actions when there was no need for action - chasing down the wrong goal that yields no result. For instance, some individuals abandon their value systems and drift significantly off-course in search for more social media 'likes' which sometimes does not translate directly to impact / relevance. Career development is not left out, such as the quest after multiple trainings without any visible impacts of such training on the work outputs and overall development of the individual. From a spiritual perspectives, true walk and communion with our Creator and the transformational benefits that come from that fellowship is sometimes ignored in favor of the number of meetings and religious activities such a person attends.

.....false measurements can also create an alarm that probably sends you down a rabbit hole and distracts from other key actions when there was no need for action.....

So, how then do you know that the measurement is wrong?

How can you tell that the indicators are pseudo-performance indicators instead of the key performance indicators? Let's explore 2 key issues that indicate faulty measurements:

The first indication of a faulty measurement system is weak or no linkage between the measurement process and the goals of the measured system indicative of a wrong measurement methodology. This can result from a copy and paste approach to measurement where what seems to be working for one process is transferred to another without a proper evaluation of the goals and context. Is what you are measuring really aligned with the goals of the system? For instance, a person who is looking to loose weight might need to measure a different set of parameters than someone else whose goal is to keep fit or worse still build some muscle and add weight. Hence, a number of people may be exercising but they may have different goals and so the measurement modes are not replicable across board. In other cases, the issues might arise from an assumption of causality whereas what exists is only correlation or a partial measurement instead of a systemic approach. For instance , monitoring exercise with no attention to food intake may result in overall suboptimal weight loss outcomes. Hence, monitoring outputs or activities may not always be strongly linked to outcomes if key indices are missed in the overall contributions to the goals.

In other cases, the issues might arise from an assumption of causality whereas what exists is only correlation...

The second indicator of faulty measurement is when the results do not match other indications of actual performance, the tell-tale signs of a broken measurement system. There is that nagging feeling of "this doesn't make sense" with so many unexplained loose ends. Similar to a pilot with faulty cockpit instrument showing that the flight is progressing well, yet the years of experience and some unusual alarms going off indicate that there is a major issue with the aircraft. The problem with broken measurement systems is that they have worked well at some point and become so trusted to the detriment of sound judgement at the point of their failures. Performance indicators that worked well at the inception of the company may now become broken when the customer base and span of stakeholders have grown and so while the limited group that is sampled for surveys appear satisfied, it doesn't appear to be translating to the sales numbers or bottomline of the company. Much like an individual with tons of social media followers unable to get participation at a paid event that they set up.

The problem with broken measurement systems is that they ............become so trusted to the detriment of sound judgement at the point of their failures...

Why might a measurement system be faulty?

One reason could be that (1) the measurement system is no longer connected to the process being monitored as seen in process plants where the connection between the visible instrument seen and the process being measured may have failed or become blocked so its no longer a representation of the system like it was in the past. In the family context, the communication lines between a parent and a child may have been blocked and as such the smiles and brief greetings that were once relics of a healthy relationship may now be a facade covering up troubled minds.

Also, (2) the measurement system may be stuck at the same point or in the past like a broken watch that appears to be right at least twice a day. Once in a while, the measurements may correlate with the reality of the situation but most times it is wrong. Sometimes we are stuck celebrating past victories that are no indications of preparedness to tackle the new challenges ahead like a driver speeding forward with his gaze permanently fixed on the rear-view mirror.

At other times, (3) it may be because the measurement system has become obsolete for the dynamic process that is being measured. Hence, such measurement systems are not sophisticated enough to account for the nuisances of the evolved process. The measurement system of the health of a young marriage that is not upscaled to account for the passage of years, new self and partner discoveries, introduction of children, amongst others may be reason why one spouse is blindsided until there is a sudden lash out from the other whose requirements for satisfaction has become more sophisticated over time.

Lastly, (4) The error margin may be too high in which case noise impacts the accuracy of the measurement. An interesting example is with stretchy fabrics that allow an individual keep wearing them comfortably even after adding a couple of inches giving a false sense of assurance of "after all my cloths still fit" until it is exposed by the non-stretchy equivalent at the same size that can't get through the arms. Imagine how disastrous it could be if a life and death decision of a doctor is based on measurements from an instrument with such a wide error margin that there may so many false positives or negatives. Yet, some of us measure our life decisions with systems that produce such a wide range of results that may be a far-cry from the true state of the situation.

Sometimes we are stuck celebrating past victories ........ like a driver speeding forward with his gaze permanently fixed on the rear-view mirror......

How then can we fix a faulty measurement system?...

In order to address some of the issues that lead to inaccurate measurements, four steps are proposed: (1) Clearly define the goals of the system/process and peel back the onions intentionally to link the goals to specific key performance indicators that can be measured. (2) Understanding that the performance context is key. What works in one industry / process may need tweaks before it can be applied to another based on the contextual differences. (3) Triangulate your measurement systems with other modes of measuring performance to ensure that it is not faulty. Calibrate accordingly based on the findings of such comparisons. (4) Complete the feedback loop by checking in consistently to review efforts vs impacts. For instance, while the results of the weighing scale is the ultimate indicator of success or not of a weightloss program, it is not realistic to carry around a bathroom scale so the proxy becomes the activity monitors. However, every day or couple of days, the successes represented in the activity monitors must be verified by the amount of weight lost on the bathroom scale. Similarly, businesses need to check-in regularly to verify that their incentive structures translate to employee experience and productivity that ultimately impacts the bottom line.

What works in one industry / process may need tweaks before it can be applied to another based on the contextual differences.......

So, bringing it all together....

Whether it's a business, career, relationship, spiritual wellbeing, or impact goals that you are driving performance towards, a few critical questions must be answered regarding your measurement systems: What are you measuring? Is your measurement consistent with your goals? Are you measuring to celebrate your past victories without looking forward? Do your measurements provide insights on how and what to improve? Does your measurement drive accountability or are you playing it safe? Are your measurement modes aligned? Is the measurement systemic or focusing on one aspect at the detriment of others? Is there too much allowance or error margin baked into your measurement system? Are you focused on efforts or results? Do you regularly stop to check if your measurements and results match before you are too far gone?

These key litmus tests help to keep your measurements from becoming pseudo indicators of progress but instead ensuring that they are focused on Key Performance Indicators (KPIs) that drive true performance.

Wishing you a transformational journey ahead

Funmilola (Funmi)

#funmilolaasasblog #pseudoperformanceindicators #intentionalmeasurements #systemthinking #credibility #faultymeasurementsystems #trueperformance #progress #results #outcomes #KPI #KeyPerformanceIndicators