Mistrust of Talent Measurement

In many organizations, you’ll find fear and mistrust. You’ll also find that employees often focus these negative sentiments on the organization’s measurement system.

A measurement system won’t work without trust. Trust is the basis of functioning human communication, and as I’ve discussed in previous posts, measurement is best understood as a means of communication. In this blog post we’ll explore these issues, and look at how to increase trust in measurement.

I had a client with a world-class performance management system. The system included many best practices:

• Individual goals cascading from the strategy of the organization
• Employees and managers met at the beginning of the year to discuss goals
• Coaching happened throughout the year, with a formal documentation period
• Final evaluation happened at the end of the year, with ratings of behavioral competencies as well as objective performance metrics.

The  employees hated the evaluation system. Why? They didn’t trust it. This organizations performance management system, like many, was a de-motivator!  

When there is no trust in measurement, the results are often ugly. In pre-revolution feudal France, the physical measurement system was controlled by the aristocracy. Measurement was idiosyncratic, fragmented, and non-standardized; there were approximately 14,000 different units of physical measurement. Mistrust of measurement, or measurement-based decisions (such as those involved in commerce), was one of the factors exacerbating class tensions.

There are historians who report metric riots over measurement issues during this period. When the nobility surrendered their privileges after the storming of the Bastille, these privileges included giving up control of measures.

This may sound extreme, but is it? What if you couldn’t trust that a pound was a pound, and the person measuring your pound of pasta was defining the measurement?

Today, we assume that physical measurements are accurate and standardized—everything from a pound of pasta to land surveys. There are governmental bodies regulating physical measures. We rarely even think about the physical measurements that we use daily.

The same is not true of talent measures. Each organization has its own set of talent measures (for good reason), and these measures are typically controlled by management.

As in the French revolution, those who control the measures have a significant source of power to use and, in some cases, abuse. Before the revolution, serfs often felt they were being cheated by nobility.  They probably were. After the revolution, there was a movement to base measurement on universal, naturally occurring objects. This is the origin of the metric system.

How does this relate to my client with the world-class performance management system? Employees hated the system because it was unpredictable and apparently arbitrary. It was unintentionally failing the fairness test in two ways. 

First, employees saw the performance appraisal procedure as unfair because it was constantly changing. In an attempt to improve the system, senior management was constantly tweaking it. Supervisors and employees weren’t sure how the system would ultimately appear on the intranet. Little was done to communicate how the system worked, or why it was changed. Given the complexities of the measures and the misunderstanding of the process, it’s understandable that employees started to wonder whether something nefarious was going on. Sometimes the tweaks made winners and losers. As a result, a near revolution was brewing in what Elton Mayo would have referred to as a social system. 

Second, employees saw the system as unfair because of the way rewards were distributed. The pool of bonus money was spread among the employees according to the ratings, often as the final ratings were calculated. This is not an unusual profit-sharing plan. Unfortunately, there were last-minute changes, sometimes for departments as a whole, that had unexpected effects on ratings and compensation.

Stability and transparency is needed to make a performance management system trusted and functional. This requires an acceptance of the imperfections of measurement, and much more communication. To a large degree, tweaking will not improve a talent measurement system—it will only serve to further distort the meaning of the measures. Often, tweaks occur because we believe in the myths of measurement:

• My mental model of performance is correct
• Measures are real
• There is a perfect measure
• People and the measurement system do not change in the process of measurement.

Opposition to Measurement

This brings up a second puzzle for a measurement guy like me. Why are so many employees, and unions, opposed to measurement?

Coming out of graduate school, I understood the formal side of measurement very well. I was surprised at the level of animosity toward measurement. I naively assumed that the honesty and accuracy associated with good measurement would enhance relationships, including employee and labor relations. I soon learned that in practice there is little appetite for measures, mostly because of how they have been, and are, used in organizations. 

I’ve found four major reasons for this opposition:

• Employees being held accountable for factors beyond their control
• Arbitrary use of power
• Differentiation between employees
• History of Taylorism.

These four reasons hamper effective talent measurement. They exert such a powerful force that new measurement initiatives start with a lack of trust. Often this lack of trust must be addressed and overcome before organizational learning and employee motivation can happen.

If you know of additional reasons, I’d love to learn about them.

Inappropriate Accountability

As I discussed in a previous post, measures are contaminated by factors beyond an individual employee’s control. For example, low-performing teammates may prevent an employee from reaching her full potential. This is, of course, a fact of life—we can’t control everything.  When employees are faced with a measurement system that ignores factors beyond their control, they can feel that their efforts are futile, and the measurement arbitrary. This tends to erode trust in the organization and management.

Arbitrary Use of Power

Managers have the power to change the measurement system; sometimes this power is used to withhold rewards or punish employees in some way. For example, if a salesperson has earned a large reward based on a previously defined incentive system, sometimes the system is reconfigured to avoid the large payout. Of course, this breaks trust—and it does happen. Unfair actions related to measurement  are remembered for a long time.

In addition, management is often responsible for interpreting or framing measures. Setting unattainable goals, for example, can lead to employee dissatisfaction and decreased morale, as employees give up trying to improve their performance.

Differentiation between Employees

Measurement is often used to differentiate between employees. If you don’t trust the measurement system to make important distinctions related to employee performance, you won’t like differentiation. 

In the coming year, many U.S. school districts will differentiate between teachers using standardized tests. Some will think this is fair, but many will not. Methods have been developed, such as value-added scores, that attempt to statistically control for factors, such as race or wealth, that affect student growth. Despite this, many fundamentally mistrust the standardized tests on which the evaluation systems rest.

Differentiating between employees is also antithetical to unionist philosophy. Unions believe that differentiation reduces solidarity, and they tend to believe that everyone is the same. In fact, this mistrust of measurement, based on valuing both similarity and solidarity, leads to a question often asked by unionists: How can we tell who is better than another? If you can’t answer this question, the only basis for differentiating payment is seniority. While experience does increase knowledge, skill, and even wisdom, it’s not performance. It is, however, an unambiguous differentiator.

History of Taylorism

In a sense, Frederick Taylor’s use of time-and-motion studies to set performance goals destroyed trust in measurement. As I discussed in the last post, Taylor’s approach to measurement was used mechanistically, against employees, and not in partnership with them.

Interestingly, Taylor assumed that employees would embrace scientific management. Sometimes they did, but most often they didn’t. He was, of course, assuming that employees were motivated only by money and the possibility of higher wages based on increased work.  Of course, we now know that this assumption is wrong.

In an interesting turn of history, employee unions and strikes were ultimately the downfall of Taylorism. Strikes by public sector employees caused Congress to hold hearings. After five years, scientific management was essentially outlawed by limiting the use of incentive wages and stopwatches.

Building Trust in Talent Measures

So how do we build trust in talent measures, given that the way measurement has been used in the past creates negative preconceptions? I have three suggestions that will help create a climate of fairness and transparency:

• Make measurement predictable. Build the best technical system you can, and accept that some people will manipulate the system. The solution to manipulation is rarely a better measure—most often, it’s better management and leadership, using the measures.
• Communicate, communicate, communicate! Communication is creating shared meaning. An organization’s definition of the meaning of measures will not be the same as employees’ meanings, or experience. Both meanings need to be acknowledged and managed in a two-way give and take. Never assume that a measure means the same thing to everyone.
• Build trust in measurement. Anything that could be interpreted as using the measures against the workforce or individuals should be avoided.  Apply measurements consistently across your organization. Hold everyone accountable to the same standards, and make those standards clear.  Remember, we want our employees to be thinking about the work and engaging with the organization.  We do not want them to be thinking, “Is this measurement system fair?”

Measuring Invisible Talent

When we think about measurement, we usually think about measuring objects—their length or weight. The concrete nature of this sort of measurement is easy to understand and predictable: A meter is always a meter.

Talent measurement, in contrast, is not easy to understand, and it’s unpredictable. Measuring talent has its challenges, but it’s one of the keys to organizational learning and employee motivation.

When it comes to measuring talent, we look for consistent and concrete measures, just as we do in the physical world. We hope for honesty in the mathematical precision offered by measures and metrics.  We hope for less wiggle room and more candor. We hope measurement data provides less theory, better insights, and obvious decisions. 

Comparing physical and talent helps us to see the value and potential of measurement. The value is high, but if we cling to the metaphor of physical measurement, we will grow frustrated. We may also miss one of the real strengths of talent measurement.

Talent measurement is different and it is complicated. Among the complications, it has a special attribute: measurement motivates. To access the benefits of talent measurement we need to consider how physical and talent measures differ. 

Talent Is Invisible

Talent measures aren’t concrete, like their physical counterparts. Many of the most important assets of today’s world are essentially invisible—think of wealth, power, relationships, personality, or intelligence.  Because the aspects of talent that we care about are invisible, it can be difficult to know what we are measuring.    

It’s not just that talent is invisible. We also need to remember that measures are just a representation of talent. This adds complications. We all know that someone’s height in inches isn’t the person. It’s easy, however, to confuse a measure of potential with the value of a high-potential employee. The measure of potential is a representation of an underlying capability, and the measure is accurate only in a probabilistic sense.   

Invisibility and representation are two reasons that talent measures tend to be less precise, and less consistent, than physical measures. Have two managers rate how well an employee completed a difficult task, and you’re likely to get two different answers. Ask two skilled carpenters to measure the length of a cabinet, and you’ll probably get two nearly identical answers, accurate to within one-sixteenth of an inch. 

Despite this, organizations often act as though their measures are nearly perfect. For example, some management consultants recommend that employees be ranked annually. Let’s be clear about what ranking actually means: Employees will be listed in order, from best to worst. To truly rank employees, there would need to be a distinction between the 10^th and 11^th best employees. Without a perfect measure, this is impossible. Since we don’t have measures that are up to this task, we would have to use other means to rank employees, such as intuition. 

Ranking employees by a physical measure sounds easier, but even this is complicated. Let’s say you’re at a family gathering and you’re taking a photo of your grandparents, siblings, nieces, nephews, and so forth. You’d ask people to organize themselves by height—shorter people in front—so that the camera can capture their faces. We often think about employees this way: We can just line them up according to some feature, such as performance. We’ll keep the best, or the tallest, and get rid of the rest.

Of course, it’s not that simple! Even physical measurement is imperfect. Imagine trying to get 1,000 employees to stand in order, by height. I can hear the questions already. Does big hair count? Should we take our shoes off? In the end, we’d probably need to ask ourselves, "what is employee height, anyway?"

If we take physical measurement to this logical conclusion, it provides a useful lesson: measurement is more complicated in practice than in theory. We may think we understand what leadership is. When it comes to measuring it, we need to get pretty specific in our meaning.  

People React to Being Measured

In general, physical measurement has few side effects. If you measure the length of a cabinet, you don’t affect the cabinet, and the cabinet isn’t likely to react. Talent, unlike inanimate objects, is affected in complicated ways by measurement. People react to measures.

In fairness, some physical objects are affected by measurement. When checking tire pressure, a small amount of air escapes. This affects the tire pressure. This is a simple example of an observer effect, which has been well documented in physics. For example, a glass thermometer absorbs thermal energy when taking a measurement. 

Observer effects on physical objects are generally unsurprising and small. Talent’s reaction to measurement is complicated and can be large. 

The effect can be positive. Measurement can lead to motivation, increased effort, and more focus.  Feedback and reasonable goals often lead to higher levels of performance, as we discussed in past blog posts.  

Practical experience however, shows that this isn’t always the case. Unfortunately, measurement can change talent in counterproductive ways, depending on the context. Measurement can de-motivate and distract. If measures are linked to very difficult goals, employees sometimes give up, or—even worse—lose faith in the organization and disengage. 

To make things worse, reactions to measurement can also motivate talent to corrupt or game the measures. Physical measurement never has this issue. Humans, however, have a major preoccupation with gaming measures. It’s so common that the famous methodologist Don Campbell, discussing program evaluation in 1975, described what has become known as Campbell's Law:

The more any quantitative social indicator is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.   

Given these complexities, perhaps talent management should steal the nickname “dismal science” from economics:

• The underlying dimension you wish to understand is invisible 
• Measuring talent is imprecise and probabilistic 
•  Humans react to measures, sometimes by changing themselves, and sometimes by changing the measure.

 

So, Do We Stop Measuring Talent?

It’s been said that since we can’t measure talent perfectly, we should simply give up and acknowledge defeat. For 30 years, some consultants have advocated getting rid of performance appraisals altogether. I’m amazed this impractical idea is still being considered.

If we don’t measure human performance, we lose a powerful motivational and learning tool. As with many aspects in life, we simply have to manage the dilemma and tension. 

Measurement is flawed and we must use it.  We can’t hide our head in the sand and hope these flaws will go away. The flaws are inherent in measuring, especially something as complicated as talent. If we step into the real world of human idiosyncrasies, measurement is a powerful tool that can help us improve organizational performance, ensure educational excellence, and motivate personal growth. 

In the coming blog posts I will further elaborate the myths of talent measurement and how we can think more clearly for organizational learning, motivation and growth.

Charley Morrow

Employee Performance Measurement is Increasing But Will Not Increase Efficiency

Employee performance has been measured since the start of the industrial revolution in the late 18^th century. By 1910, scientific management—the attempt to improve efficiency by applying engineering principles and measurement, in manufacturing —was reaching its peak. 

Although scientific management as a school of thought had faded by the 1930s, it continues to influence the way we measure and manage, and it provides fascinating historical insights into industry. Scientific management was largely focused on per-worker output, how to increase output by finding the right employees, incentive schemes and best practices to ensure that systems function optimally. 

Does this sound familiar? It should. Management, and our society in general, continually focus on these topics. Our use of measurement has only expanded. Today, 97% of organizations have an employee appraisal process, and many organizations are working to increase the number of appraisals to several times each year.  

With current social and technological trends, I expect human performance measurement will continue to increase. I also expect that people will continue to be surprised at the real outcomes and consequences of measurement. I’ll make another prediction: We’ll see little value from all this measurement, unless we start cultivating wisdom in our use of measures.

Frederick Taylor’s Underestimated Influence

The influence of Frederick Taylor, generally considered the founder of scientific management, has been compared to Darwin and Freud.  As Taylor wrote in his 1911 best seller Scientific Management:

… the end of our coal and iron is in sight. But our larger wastes of human effort, which go on every day through such of our acts as are blundering, ill-directed, or inefficient … are less visible, less tangible, and are but vaguely appreciated. … And for this reason, even though our daily loss from this source is greater than from our waste of material things, the one has stirred us deeply, while the other has moved us but little.

Taylor is the father of modern management.  At the turn of the century, management primarily were concerned with budgeting and hiring. Teams of workers defined the tasks. Workers in a sense were still in somewhat of a guild system and had some level of autonomy. Ideas presented by Taylor, and then Henry Ford’s production line, changed things considerably.

   Frederick W. Taylor

While famous for time and motion studies, Taylor had two bigger ideas associated with measuring employee performance:

• System improvement (also known as organizational learning)
• Employee accountability (to achieve targeted levels of performance).

These ideas continue to drive improvement efforts to this day. Taylor believed that per-individual output should be measured, and that the measure should be studied to understand both the system of work and how it can be improved. He also advocated for developing challenging targets for employee output.  

Taylor’s most famous example was of moving pig-iron, which was made completely by men’s labor.  To illustrate scientific management, he described an intervention at Bethlehem steel. He found that the average man moved about 12 ½ tons of pig iron each day, but the best handlers loaded 47 ½ tons. (I find both values astounding.)  By selecting the right men for the job, specifying how the loading task should be completed, and setting appropriate goals, he was able to improve per-employee pig-iron loading output. The loading tasks were streamlined and proper rests were enforced to ensure that a high level of output could be maintained for the entire day. Pig-iron handlers were accountable to achieve the goal of 47 ½ tons per day. Failure to consistently achieve the goal would result in reassignment.

Similarly, through careful study Taylor found the optimal shovel load was 21 pounds. This finding suggested many system-wide changes—in tools, team organization, and supervision. The organization provided different shovel sizes and shapes for different material density and characteristics. Workers that had been managed as teams were now managed as individuals, each accountable for performance targets.  Management structures changed to oversee these major changes.  

While Taylor’s time-and-motion studies are no longer relevant, given robotics and increasing service work, the measurement themes continue, and the themes of system improvement (organizational learning) and employee accountability continue to this day. In fact, these are two ideas with growing currency.

Organizational learning, which goes by many names, is simply the idea that organizations are entities that can learn and adapt. By looking deeply at the organization’s environment, processes, structures, routines, and culture, we can understand and improve them. There are many methods for encouraging organizational learning.  Whole conferences and societies have been developed around organizational learning, feedback, and measurement. In many situations, measurement of employee performance is the basis of organizational learning. 

Given the increased need to adapt organizations in our changing political, economic, social, and technical environment, I expect that measurement will increase. I worry, however, that organizations do not think clearly about the performance measures they track.

As organizations grow more complicated and employees more specialized, accountability for results has replaced the idea of managing tasks. Taylor was concerned with a moving pig-iron and micro-tasks such as lifting and carrying between stations. Now an employee is likely to have more complicated assignments comprised of dozens of tasks (for example, creating a new software feature or maintaining a database). Now we measure value-added and organizational results.  

Today, the essence of progressive management thinking could be stated as, “I do not want to micro-manage, so I am going to assign accountability for results and let my employees use their creativity and capability to get the job done.  In order to hold them accountable, I am going to measure their performance or productivity.” Many management theorists, such as Daniel Pink, argue for a result-oriented/only work environment (ROWE).  (Note that Taylor was indeed a micro-manager, so the parallel ends quickly.)  

I am a fan of managing results, not tasks. If you have ever been micro-managed, you’re probably a fan of managing results as well. I’m not completely sure, however, that we have the expertise to make it work. I see many organizations struggle with measures of performance—with pushback from employees and a blind push from management. ROWE works best when there are clear measures of performance that are linked to valued organizational outcomes.  These measures do not exist in many situations-- for example in support functions.

The final driver of increased measurement of employee performance is the computer-based work environment. Due to increasingly inexpensive databases, this environment allows more measurement and data-tracking. As computers facilitate more work, keystrokes, conversations, tasks, and transactions can be—and often are—automatically recorded. These digital trails can be compiled very cheaply. Footsteps, driving routes, and bathroom breaks can all be easily recorded with easily available technologies. It is inevitable that new measures of performance will be dreamed up, developed, and implemented. 

What Have We Learned About Managing and Leading With Measures?

My concern is our ability to manage, learn from, and improve from all this measurement. If we’re not careful, the additional performance measurement will only lead to additional misunderstanding and organizational chaos, misalignment, gaming of the system, and, in general, dysfunctional behavior. I’m not sure that our wisdom has kept pace with the amount of data and performance measures.  

In fact, problems associated with employee performance measurement appeared soon after the advent of scientific management.  Measuring employee performance often leads to strange outcomes. 

This is demonstrated by what’s known as the Hawthorn Effect, which showed, as far back as the mid-1920s, that when you measure employees’ performance, they react. Sometimes they react positively, sometimes negatively. Not surprisingly, it can be difficult to predict which outcome you will find.

The Hawthorn plant manufactured telephone equipment, and was attempting to identify the best practices to increase employee output. In one study, they focused on lighting. They increased the brightness of the lights in the plant, and employee output increased. They dimmed the lights, and, surprise—the same thing happened. Employee output increased whether the lights were brightened or dimmed. It wasn’t the quality of the lighting, but the change that employees were reacting to.

Additional studies from the Hawthorne plant, referred to as the "bank wiring room studies," revealed that social forces were affecting the output. Employees were aware of the measure and how they were studied by management. As a result the employees used the measure to communicate to management that they were, in fact, doing a reasonable job. Concerns, such as layoffs based on the measures, affected employee output. In these studies, employees attempted to communicate to management that they were working in a steady and reasonable manner.  

These studies have made it clear that measurement does not cause efficiency —there is always an underlying and very human mechanism that is a reaction to measurement. The reaction may or may not be improvement. These very human processes hinge on reactions to the measures that include communication and messaging, image management, and social interaction.

In day-to-day work environments, we continue to ignore what studies published 90 years ago clearly demonstrated. Many attempt to improve performance with measures, without giving adequate thought to managing the underlying human processes that will lead to improvement. 

Consider the testing mania present in American public schools.  These schools are expected to improve, largely because they are being measured and expected to improve. There is a blind belief in accountability. 

Similarly, consider the 97% of organizations that have performance appraisal systems that attempt to provide feedback to individuals in the hope that it will automatically improve performance, learning, and motivation.

Unknown to most, the ghost of Fredrick Taylor is alive in many of our modern organizations. Blind beliefs in simplistic ideas about measurement continue. We need to use measurement to increase efficiency, but a naive belief that productivity will increase without addressing the very human element of work is defeating the purpose, and the effectiveness, of the measures.

If you have examples of simplistic thinking behind employee performance measurement, I’d love to hear your story.

Charley Morrow