Why hiring the ‘best’ people produces the least creative results

From Aeon:

While in graduate school in mathematics at the University of Wisconsin-Madison, I took a logic course from David Griffeath. The class was fun. Griffeath brought a playfulness and openness to problems. Much to my delight, about a decade later, I ran into him at a conference on traffic models. During a presentation on computational models of traffic jams, his hand went up. I wondered what Griffeath – a mathematical logician – would have to say about traffic jams. He did not disappoint. Without even a hint of excitement in his voice, he said: ‘If you are modelling a traffic jam, you should just keep track of the non-cars.’

The collective response followed the familiar pattern when someone drops an unexpected, but once stated, obvious idea: a puzzled silence, giving way to a roomful of nodding heads and smiles. Nothing else needed to be said.

Griffeath had made a brilliant observation. During a traffic jam, most of the spaces on the road are filled with cars. Modelling each car takes up an enormous amount of memory. Keeping track of the empty spaces instead would use less memory – in fact almost none. Furthermore, the dynamics of the non-cars might be more amenable to analysis.

Versions of this story occur routinely at academic conferences, in research laboratories or policy meetings, within design groups, and in strategic brainstorming sessions. They share three characteristics. First, the problems are complex: they concern high-dimensional contexts that are difficult to explain, engineer, evolve or predict. Second, the breakthrough ideas do not arise by magic, nor are they constructed anew from whole cloth. They take an existing idea, insight, trick or rule, and apply it in a novel way, or they combine ideas – like Apple’s breakthrough repurposing of the touchscreen technology. In Griffeath’s case, he applied a concept from information theory: minimum description length. Fewer words are required to say ‘No-L’ than to list ‘ABCDEFGHIJKMNOPQRSTUVWXYZ’. I should add that these new ideas typically produce modest gains. But, collectively, they can have large effects. Progress occurs as much through sequences of small steps as through giant leaps.

Third, these ideas are birthed in group settings. One person presents her perspective on a problem, describes an approach to finding a solution or identifies a sticking point, and a second person makes a suggestion or knows a workaround. The late computer scientist John Holland commonly asked: ‘Have you thought about this as a Markov process, with a set of states and transition between those states?’ That query would force the presenter to define states. That simple act would often lead to an insight.

The burgeoning of teams – most academic research is now done in teams, as is most investing and even most songwriting (at least for the good songs) – tracks the growing complexity of our world. We used to build roads from A to B. Now we construct transportation infrastructure with environmental, social, economic and political impacts.

The complexity of modern problems often precludes any one person from fully understanding them. Factors contributing to rising obesity levels, for example, include transportation systems and infrastructure, media, convenience foods, changing social norms, human biology and psychological factors. Designing an aircraft carrier, to take another example, requires knowledge of nuclear engineering, naval architecture, metallurgy, hydrodynamics, information systems, military protocols, the exercise of modern warfare and, given the long building time, the ability to predict trends in weapon systems.

. . . .

The multidimensional or layered character of complex problems also undermines the principle of meritocracy: the idea that the ‘best person’ should be hired. There is no best person. When putting together an oncological research team, a biotech company such as Gilead or Genentech would not construct a multiple-choice test and hire the top scorers, or hire people whose resumes score highest according to some performance criteria. Instead, they would seek diversity. They would build a team of people who bring diverse knowledge bases, tools and analytic skills. That team would more likely than not include mathematicians (though not logicians such as Griffeath). And the mathematicians would likely study dynamical systems and differential equations.

. . . .

Even with a knowledge domain, no test or criteria applied to individuals will produce the best team. Each of these domains possesses such depth and breadth, that no test can exist. Consider the field of neuroscience. Upwards of 50,000 papers were published last year covering various techniques, domains of enquiry and levels of analysis, ranging from molecules and synapses up through networks of neurons. Given that complexity, any attempt to rank a collection of neuroscientists from best to worst, as if they were competitors in the 50-metre butterfly, must fail. What could be true is that given a specific task and the composition of a particular team, one scientist would be more likely to contribute than another. Optimal hiring depends on context. Optimal teams will be diverse.

Link to the rest at Aeon

A reminder that PG doesn’t always agree with items he posts on TPV.

In recent years, PG has become increasingly suspicious of any recommendation for “diversity” since the term has picked up so much political baggage.

He might even suggest limiting “diversity” to a term of art which stands solely in the realm of the politically defined terms that mean whatever the politics of the moment say they mean.

Regarding the OP, if “diversity” means a collection of intelligent and creative individuals with a variety of different skills and bases of knowledge, PG thinks such a group can work effectively on complex problems.

However, the mental and social overhead that accompanies the operations of groups of people can make the process of accomplishing truly creative work difficult and unlikely. On more than one occasion in PG’s experience, a single person has made doing useful work in a group difficult or impossible. Sometimes, this problem is not a factor of background or intelligence so much as it is a matter of the personality of the individual.

Teams can be a useful work environment or a giant waste of time depending upon the personalities, experiences, talents and abilities of the team members. Group dynamics does usually operate in such a way that conventional solutions or extrapolations from past solutions into a changed environment are a more likely product of teams than really breakthrough ideas.

On a couple of occasions during his professional and business career, PG has had the privilege of working with individuals who were bonafide geniuses in their fields with a track record of technical breakthroughs that was astounding. From this admittedly limited group of examples, PG can’t imagine these people working well with a team nor can he imagine a team producing the types of brilliant innovations that the individuals did, certainly not within the timeframe during which the individuals were making their discoveries and creating their innovations.

10 thoughts on “Why hiring the ‘best’ people produces the least creative results”

  1. The multidimensional or layered character of complex problems also undermines the principle of meritocracy: the idea that the ‘best person’ should be hired.

    Of course you should hire the best person. Consider the alternative.

  2. I agree that diversity is often used as an excuse, but I think you miss the main point of the article:

    Corporations, non-profits, governments, universities and even preschools test, score and hire the ‘best’. This all but guarantees not creating the best team. Ranking people by common criteria produces homogeneity.

    The issue is that we do not really have an objective way to measure merit (except in fields like sport), therefore all “bests” are somewhat arbitrary. So, when you choose only the “best people” you are effectively chosing people that are best at one specific thing, but they may lack in other aspects.

    • therefore all “bests” are somewhat arbitrary. So, when you choose only the “best people” you are effectively chosing people that are best at one specific thing, but they may lack in other aspects.

      So, baseball managers recruit nine pitchers because they are best at one thing?

      • It seems to me that you purposefully nitpicking with this comment. I do not think is controversial to say that we do not really have a definitive way to perfectly rank the ability of a person to do a job.

        We do have only methods that works partially. In other words, our ranking methods do not have the precision to correctly identify “the best”, but only the “reasonably better than average”. So, to me it makes sense to admit that there is room for interpretation and use a mix of different criteria to select somebody.

        • Either people work out, and participate in a harmonious team, or you have hired someone who makes things difficult for others in search of improving their own status.

          The top positions, the top salaries go to someone. That someone is not always qualified to do the job, but is often qualified to fight for the job better because that’s their sole criterion.

          Unfortunately, the ‘real’ person doesn’t show up during the interview process, and is never going to be selected via algorithm from digitized resumes. Some people just learn early how to exaggerate their accomplishments. Some of your bad interviewers are also people who work very hard to make sure the work gets done – precisely because they spend their time working instead of schmoozing.

          It’s an imperfect world. Some workplaces are toxic. Some people are con artists.

          • The costs of hiring the wrong person, particularly if that person is harboring and hiding a toxic personality, can be a very expensive mistake.

        • Didn’t have to nitpick it. That work was already done.

          I agree we don’t have perfect methods. However, I disagree that we are limited to picking people who are good at one specific thing. The best person can be someone who is good at multiple things. It can be someone who is best at nothing, but very good at a few things. It can be the person with a track record of success regardless of any ranking method.

          Assembling a team does not mean one has to use the same standard for each member. There is often a very different set of information available for candidates. The best candidate is the one who best fills out the team, not the one who falls somewhere on some ranking system.

          When I finished school I got a great break. The line manager who was hiring was a Marine Corps vet. So was I, and I was the only one. He called me directly, and I was hired the next day. Best candidate?

          • I agree with your opinion, but this is exactly what the article states when it says:

            Ranking people by common criteria produces homogeneity.

            While when you were saying:

            Of course you should hire the best person. Consider the alternative.

            you were implying that there is one best person, which is different from what you are saying right now that there is one best team, i.e., one group of people assembled with different somewhat-flawed standards.

            The article is also saying we cannot really hope to find the “objective best” (i.e., the meritocracy). Which seems to be what PG disagrees with, or at least this is my interpretation of his comment. Instead this is what I was defending when referring to “one thing”, that is to say “one set of criteria”.

            So we can call it nitpicking or misunderstanding, but it seems that we fundamentally agree and we are just debating terms.

            • you were implying that there is one best person, which is different from what you are saying right now that there is one best team, i.e., one group of people assembled with different somewhat-flawed standards.

              There is a best person, and managers routinely use lots of methods to find him. But, there is no single ranking method that identifies him. The evaluation method is a function of the position being filled.

              Choosing the best person definitely does not mean “effectively chosing people that are best at one specific thing.”

  3. I’ve been in, created, and seen a lot of teams in smallish businesses, esp. related to computers and software.

    IMHO, you want a mix of viewpoints (different experiences/specialties), sturdy-enough personalities that are not shy about venturing & defending opinions, and a “let’s explore that concept for a minute” group curiosity dynamic.

    But the most important things are:

    1) A leader who’s willing to spend resources on ideas without suppressing tentative suggestions until they’ve been explored, and who can get guests in to comment on feasibility within a specific organization.
    2) No assholes who “won’t play nice” (within reason — tough/sharp/challenging is fine, bullying/meanness is not, being dysfunctional)
    3) A common understanding of (and at least some commitment to) the fact that the team was built for a point — improving the business/technology they’re involved in, not just swanning around with fantasies.

Comments are closed.

Secured By miniOrange