Measuring Invisible Talent

When we think about measurement, we usually think about measuring objects—their length or weight. The concrete nature of this sort of measurement is easy to understand and predictable: A meter is always a meter.

Talent measurement, in contrast, is not easy to understand, and it’s unpredictable. Measuring talent has its challenges, but it’s one of the keys to organizational learning and employee motivation.

When it comes to measuring talent, we look for consistent and concrete measures, just as we do in the physical world. We hope for honesty in the mathematical precision offered by measures and metrics.  We hope for less wiggle room and more candor. We hope measurement data provides less theory, better insights, and obvious decisions. 

Comparing physical and talent helps us to see the value and potential of measurement. The value is high, but if we cling to the metaphor of physical measurement, we will grow frustrated. We may also miss one of the real strengths of talent measurement.

Talent measurement is different and it is complicated. Among the complications, it has a special attribute: measurement motivates. To access the benefits of talent measurement we need to consider how physical and talent measures differ. 

Talent Is Invisible

Talent measures aren’t concrete, like their physical counterparts. Many of the most important assets of today’s world are essentially invisible—think of wealth, power, relationships, personality, or intelligence.  Because the aspects of talent that we care about are invisible, it can be difficult to know what we are measuring.    

It’s not just that talent is invisible. We also need to remember that measures are just a representation of talent. This adds complications. We all know that someone’s height in inches isn’t the person. It’s easy, however, to confuse a measure of potential with the value of a high-potential employee. The measure of potential is a representation of an underlying capability, and the measure is accurate only in a probabilistic sense.   

Invisibility and representation are two reasons that talent measures tend to be less precise, and less consistent, than physical measures. Have two managers rate how well an employee completed a difficult task, and you’re likely to get two different answers. Ask two skilled carpenters to measure the length of a cabinet, and you’ll probably get two nearly identical answers, accurate to within one-sixteenth of an inch. 

Despite this, organizations often act as though their measures are nearly perfect. For example, some management consultants recommend that employees be ranked annually. Let’s be clear about what ranking actually means: Employees will be listed in order, from best to worst. To truly rank employees, there would need to be a distinction between the 10^th and 11^th best employees. Without a perfect measure, this is impossible. Since we don’t have measures that are up to this task, we would have to use other means to rank employees, such as intuition. 

Ranking employees by a physical measure sounds easier, but even this is complicated. Let’s say you’re at a family gathering and you’re taking a photo of your grandparents, siblings, nieces, nephews, and so forth. You’d ask people to organize themselves by height—shorter people in front—so that the camera can capture their faces. We often think about employees this way: We can just line them up according to some feature, such as performance. We’ll keep the best, or the tallest, and get rid of the rest.

Of course, it’s not that simple! Even physical measurement is imperfect. Imagine trying to get 1,000 employees to stand in order, by height. I can hear the questions already. Does big hair count? Should we take our shoes off? In the end, we’d probably need to ask ourselves, "what is employee height, anyway?"

If we take physical measurement to this logical conclusion, it provides a useful lesson: measurement is more complicated in practice than in theory. We may think we understand what leadership is. When it comes to measuring it, we need to get pretty specific in our meaning.  

People React to Being Measured

In general, physical measurement has few side effects. If you measure the length of a cabinet, you don’t affect the cabinet, and the cabinet isn’t likely to react. Talent, unlike inanimate objects, is affected in complicated ways by measurement. People react to measures.

In fairness, some physical objects are affected by measurement. When checking tire pressure, a small amount of air escapes. This affects the tire pressure. This is a simple example of an observer effect, which has been well documented in physics. For example, a glass thermometer absorbs thermal energy when taking a measurement. 

Observer effects on physical objects are generally unsurprising and small. Talent’s reaction to measurement is complicated and can be large. 

The effect can be positive. Measurement can lead to motivation, increased effort, and more focus.  Feedback and reasonable goals often lead to higher levels of performance, as we discussed in past blog posts.  

Practical experience however, shows that this isn’t always the case. Unfortunately, measurement can change talent in counterproductive ways, depending on the context. Measurement can de-motivate and distract. If measures are linked to very difficult goals, employees sometimes give up, or—even worse—lose faith in the organization and disengage. 

To make things worse, reactions to measurement can also motivate talent to corrupt or game the measures. Physical measurement never has this issue. Humans, however, have a major preoccupation with gaming measures. It’s so common that the famous methodologist Don Campbell, discussing program evaluation in 1975, described what has become known as Campbell's Law:

The more any quantitative social indicator is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.   

Given these complexities, perhaps talent management should steal the nickname “dismal science” from economics:

• The underlying dimension you wish to understand is invisible 
• Measuring talent is imprecise and probabilistic 
•  Humans react to measures, sometimes by changing themselves, and sometimes by changing the measure.

 

So, Do We Stop Measuring Talent?

It’s been said that since we can’t measure talent perfectly, we should simply give up and acknowledge defeat. For 30 years, some consultants have advocated getting rid of performance appraisals altogether. I’m amazed this impractical idea is still being considered.

If we don’t measure human performance, we lose a powerful motivational and learning tool. As with many aspects in life, we simply have to manage the dilemma and tension. 

Measurement is flawed and we must use it.  We can’t hide our head in the sand and hope these flaws will go away. The flaws are inherent in measuring, especially something as complicated as talent. If we step into the real world of human idiosyncrasies, measurement is a powerful tool that can help us improve organizational performance, ensure educational excellence, and motivate personal growth. 

In the coming blog posts I will further elaborate the myths of talent measurement and how we can think more clearly for organizational learning, motivation and growth.

Charley Morrow

Measuring Employee Engagement

As employers realize that they have “engagement issues,” many are trying to quickly assess the state of their workforce. Many will buy a “ready to go” survey that includes an engagement index. This is fine, but there are lots of options.  I will write about a few.

What sort of engagement is important for your organization?
There are many types of engagement. Consider, does your organization need better engagement with safety, with customers or with the mission of the organization? I have an engineer friend who tells me he always is engaged.  He loves his profession designing manufacturing controllers, but he has very stormy relationships with employers. As a result, this skilled and valuable engineer is constantly changing jobs.  If this personable guy had closer relationships with his boss and co-workers he might stick around. A focus on retention is missing in his employers. Be clear on what is strategically important to your organization.

Your organization may be more interested in engaging certain types of employees—high potentials, hard-to-staff skill groups, or generational groups (e.g., Boomers, Millennials). Knowing what is strategically important to your organization is critical to measuring engagement.

Surveys
Most organizations will choose a survey to understand workforce engagement. At first blush, this is an easy choice. Surveys results are easy to stratify by demographic category (such as age, education-level, or department). These stratifications allow fine grained review and comparison of different groups. This helps to target intervention and to provide useful feedback to managers.

Surveys, however, do require expertise to write, implement and interpret. The difficulty in developing surveys goes beyond the technical aspects of writing and implementing surveys. Surveys are difficult to write because they require the developer to understand the dimensions of workforce sentiment that are important to employees and the strategy well enough to ask intelligent questions.

“Census” surveys, which invite the entire population of employees to take part, are most common. In addition to the time it takes to have all employees take part in the survey, asking about engagement can raise employee expectations that “something is wrong,” or “something is going to change” around here. It is critical to act upon survey results; raised expectations can actually reduce engagement!

Standardized surveys, such as Gallup’s Q12, the Great Places To Work Institute’s Survey and Sage Assessment’s REALI-Index are easy solutions in that they are research based and have evolved through use in multiple organizations. They often include norms that allow you to compare your organization to others. Ultimately, comparisons within your company are more useful, but people are often curious about “how do we compare?”

Successful organizations have a unique strategy, which suggests a need to develop your own survey. Yet, this is time consuming and requires experience and skill in developing surveys. Perhaps the best balancing act is to use a standard survey and add a handful of custom questions to ensure that your specific culture and strategy are included in the survey.

Focus groups
Compared to surveys, focus groups are blunt instruments. But, not all projects call for razor sharp precision—sometimes a more general tool is needed. Stratification and contrasts are not really possible with focus groups, rather you learn about general sentiments. While surveys do extremely well at slicing data, focus groups allow a nuanced understanding of the issues that matter to employees. Focus groups can be easy to conduct-- and can give you a quick directional sense of engagement.

Surveys are primarily static questions to which employees respond. Focus groups are more generative— groups of employees can frame the issues that matter to them.

Years ago I ran focus groups at a large organization with very high engagement. Most employees in this pharmaceutical company truly believed they were working in alignment with the words of one of the founders: “… medicine is for the patient. It is not for the profits...” While consulting, I was genuinely impressed by the engagement in the organization. It was a joy to be in this organization—employees were excited about their work and excited about their organization.

In the midst of the scheduled focus groups, however, a key product was pulled from the market.  The media suggested the organization knowingly endangered patient safety by recklessly selling the product despite knowledge of deadly side-effects. The impact was immediately obvious in the focus groups. Focus groups were markedly different before and after the recall.  Individuals whose self concept was built on working for an ethical company were wracked with doubt. Suddenly, employees in focus groups started doubting the company, distrusting management and disengaging. Concerns about management were voiced.  The abrupt changes provided insights into how to proceed with a culture change initiative. I learned that informal focus groups tell you lots about engagement and they provide you with insights that you would never learn from a survey.

You can run a focus group of the key talent group to understand the engagement of small fractions of employees. This will provide you with insights into specific strata without a survey.

Remember Engagement, Not Measurement, Matters
Regardless of how you do it, I encourage you to understand engagement in your organization. I’m a measurement guy but I have to say the goal is not measurement.  The goal is to understand and improve engagement.  It is a mistake to focus too much on measurement perfection—what matters is engagement.

I’d like to hear about how you are measuring engagement. Please comment!

Charley Morrow
www.sageassessments.com