Measuring Invisible Talent

When we think about measurement, we usually think about measuring objects—their length or weight. The concrete nature of this sort of measurement is easy to understand and predictable: A meter is always a meter.

Talent measurement, in contrast, is not easy to understand, and it’s unpredictable. Measuring talent has its challenges, but it’s one of the keys to organizational learning and employee motivation.

When it comes to measuring talent, we look for consistent and concrete measures, just as we do in the physical world. We hope for honesty in the mathematical precision offered by measures and metrics.  We hope for less wiggle room and more candor. We hope measurement data provides less theory, better insights, and obvious decisions. 

Comparing physical and talent helps us to see the value and potential of measurement. The value is high, but if we cling to the metaphor of physical measurement, we will grow frustrated. We may also miss one of the real strengths of talent measurement.

Talent measurement is different and it is complicated. Among the complications, it has a special attribute: measurement motivates. To access the benefits of talent measurement we need to consider how physical and talent measures differ. 

Talent Is Invisible

Talent measures aren’t concrete, like their physical counterparts. Many of the most important assets of today’s world are essentially invisible—think of wealth, power, relationships, personality, or intelligence.  Because the aspects of talent that we care about are invisible, it can be difficult to know what we are measuring.    

It’s not just that talent is invisible. We also need to remember that measures are just a representation of talent. This adds complications. We all know that someone’s height in inches isn’t the person. It’s easy, however, to confuse a measure of potential with the value of a high-potential employee. The measure of potential is a representation of an underlying capability, and the measure is accurate only in a probabilistic sense.   

Invisibility and representation are two reasons that talent measures tend to be less precise, and less consistent, than physical measures. Have two managers rate how well an employee completed a difficult task, and you’re likely to get two different answers. Ask two skilled carpenters to measure the length of a cabinet, and you’ll probably get two nearly identical answers, accurate to within one-sixteenth of an inch. 

Despite this, organizations often act as though their measures are nearly perfect. For example, some management consultants recommend that employees be ranked annually. Let’s be clear about what ranking actually means: Employees will be listed in order, from best to worst. To truly rank employees, there would need to be a distinction between the 10^th and 11^th best employees. Without a perfect measure, this is impossible. Since we don’t have measures that are up to this task, we would have to use other means to rank employees, such as intuition. 

Ranking employees by a physical measure sounds easier, but even this is complicated. Let’s say you’re at a family gathering and you’re taking a photo of your grandparents, siblings, nieces, nephews, and so forth. You’d ask people to organize themselves by height—shorter people in front—so that the camera can capture their faces. We often think about employees this way: We can just line them up according to some feature, such as performance. We’ll keep the best, or the tallest, and get rid of the rest.

Of course, it’s not that simple! Even physical measurement is imperfect. Imagine trying to get 1,000 employees to stand in order, by height. I can hear the questions already. Does big hair count? Should we take our shoes off? In the end, we’d probably need to ask ourselves, "what is employee height, anyway?"

If we take physical measurement to this logical conclusion, it provides a useful lesson: measurement is more complicated in practice than in theory. We may think we understand what leadership is. When it comes to measuring it, we need to get pretty specific in our meaning.  

People React to Being Measured

In general, physical measurement has few side effects. If you measure the length of a cabinet, you don’t affect the cabinet, and the cabinet isn’t likely to react. Talent, unlike inanimate objects, is affected in complicated ways by measurement. People react to measures.

In fairness, some physical objects are affected by measurement. When checking tire pressure, a small amount of air escapes. This affects the tire pressure. This is a simple example of an observer effect, which has been well documented in physics. For example, a glass thermometer absorbs thermal energy when taking a measurement. 

Observer effects on physical objects are generally unsurprising and small. Talent’s reaction to measurement is complicated and can be large. 

The effect can be positive. Measurement can lead to motivation, increased effort, and more focus.  Feedback and reasonable goals often lead to higher levels of performance, as we discussed in past blog posts.  

Practical experience however, shows that this isn’t always the case. Unfortunately, measurement can change talent in counterproductive ways, depending on the context. Measurement can de-motivate and distract. If measures are linked to very difficult goals, employees sometimes give up, or—even worse—lose faith in the organization and disengage. 

To make things worse, reactions to measurement can also motivate talent to corrupt or game the measures. Physical measurement never has this issue. Humans, however, have a major preoccupation with gaming measures. It’s so common that the famous methodologist Don Campbell, discussing program evaluation in 1975, described what has become known as Campbell's Law:

The more any quantitative social indicator is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.   

Given these complexities, perhaps talent management should steal the nickname “dismal science” from economics:

• The underlying dimension you wish to understand is invisible 
• Measuring talent is imprecise and probabilistic 
•  Humans react to measures, sometimes by changing themselves, and sometimes by changing the measure.

 

So, Do We Stop Measuring Talent?

It’s been said that since we can’t measure talent perfectly, we should simply give up and acknowledge defeat. For 30 years, some consultants have advocated getting rid of performance appraisals altogether. I’m amazed this impractical idea is still being considered.

If we don’t measure human performance, we lose a powerful motivational and learning tool. As with many aspects in life, we simply have to manage the dilemma and tension. 

Measurement is flawed and we must use it.  We can’t hide our head in the sand and hope these flaws will go away. The flaws are inherent in measuring, especially something as complicated as talent. If we step into the real world of human idiosyncrasies, measurement is a powerful tool that can help us improve organizational performance, ensure educational excellence, and motivate personal growth. 

In the coming blog posts I will further elaborate the myths of talent measurement and how we can think more clearly for organizational learning, motivation and growth.

Charley Morrow

Human Performance Measures: Start of a Series

I’ve been working with people measures for more than 25 years. Nearly every day, I see strong reactions to these common leadership tools.  Some embrace measurement as a tool for positive change, and others are nervous. Some question the measures, and others hide behind the authority of the data. 

These reactions to measurement and data fascinate me. They also hold the key to getting results from measurement systems. 

When measurement systems work well, people develop understanding, gain insight, become motivated, and set new directions. Just as often, however, measures simply do not work. In these cases, people ignore the measures or build elaborate defenses to dodge, manipulate, or diminish the data.

Over the next few months I’ll be writing about how systems and people respond to measures of human performance and how organizations can get beyond negative reactions. This is a topic I’ve been researching for years, and it may be my strongest and most nuanced area of understanding. 

I started my career focused on measurement systems. I took enough graduate courses in statistics and methodology to work as a psychometrician, and my dissertation combined the disciplines of psychology and economics. 

As I matured and worked in the real world of organizations, I started to see that the value of measurement can be found less in precision and mathematical finesse than in communication and learning. The most elegant performance management system is useless unless it is genuinely called on to help people communicate, learn, and adapt. 

In other words, measures need to be applied to produce data; data needs to be reviewed and interpreted to be useful; and useful information needs to be considered in context if people are to learn and improve. 

I can say with confidence that measures and data alone will not change organizations or behavior. There are too many psychological, organizational, and social factors that can prevent measures from translating into learning and improvement.

As a society, we spend huge sums of money on human performance measurement—and we start measurement early. All of us are familiar with the U.S. public education system, which now tests every student in the third through eight grade annually. In a number of states, databases are being developed to link these test scores to school, teachers, and student demographic information. 

When we graduate from the public education system, we find that most large organizations rely on annual employee appraisal systems. A manager can spend a few months each year rating employees, summarizing the information, and providing feedback. 

Despite the intensity of the data-gathering, improvement is not obvious. Many are dissatisfied with the measurement systems.  As a result, these measurement systems are often re-imagined and implemented with great hope and promise, only to fail. I don’t think much of this activity and investment. Don’t misunderstand: I’m a fan of measurement, because it’s critical to precise feedback and growth. But I’m an advocate for thoughtful investment in measurement. I’ve seen its transformative power. 

The public education system is still experimenting with measurement systems, and will be for years to come. Some corporations rethink their annual appraisal systems regularly. 

Technological and social trends suggest that performance measurement will only increase. Some argue that this investment is inappropriate. Addressing the merits of this societal investment isn’t my purpose here. My purpose is to make sure that individuals, organizations, and society get more value from the investments that are made.

I have workable tools and tips to make sure all of this data yields some return. Paradoxically, I won’t spend much time writing about measures. As I’ve said, it’s not as much about the measures as how they are used. I hope you will find the posts in the following weeks useful.