Roblog

Recently, I wrote about the distinction between “puzzles” and “mysteries”. Puzzles are problems that have clear definitions, simple rules, and objective solutions. Even if you don’t know what the right answer is, you know that it exists. Mysteries, on the other hand, are problems that cannot be conclusively answered; they can only be framed in different ways, in an attempt to define their ambiguities.

To solve a puzzle, you generally add more information: do more research, experiment more, analyse more data. But mysteries have the frustratingly opposite quality. The more information you add, the murkier things become; the problem is usually not that you have too little information, but too much.

Businesses tend to treat hiring as a puzzle, as though the question of the best candidate for an open position is one that has a definite, predictable, objective answer. They set about trying to discover that answer in the way that you always solve puzzles: by adding more information. And so candidates are subjected to CV sifts, phone interviews, face-to-face interviews, psychometric tests, reference checks. The more information you have, this logic suggests, the closer you’re likely to be to the answer and the better your hire will be.

But what if that isn’t the case? What if all that extra information isn’t helping you solve a puzzle, but is actually muddling a mystery? There’s certainly a lot to suggest that this might be the case.

Teams are extraordinarily complex. A team of ten people contains ten different human beings, with ten different backstories, educations, and sets of experiences. That same team of ten people also contains 45 separate relationships between individual team members¹ – relationships with different histories and dynamics, relationships that might be more or less effective. These dynamics are hard to predict and are unstable; it’s far from clear what will happen when a team encounters different situations or when the makeup of the team changes.

In fact, this instability is such that it might not even make sense to think of teams as something that can change. Instead, we might do well to see each addition or removal of a team member as creating a whole new team, as Marcus Hammarberg explains:²

“A team is a immutable structure – if you remove or add a new member to a team it’s not the same team. It’s a new team. They need to storm, form and norm all over again. It might be faster than if everyone is new but it still happens.”

In other words, hiring someone into a team requires the team to go through the same stages that a new team would go through: feeling out each others’ skills and relative statuses; gaining each others’ trust; developing intimacy and comfort with one another; and finally working effectively towards a common goal. It’s a huge change, and the nature and impact of that change on the team is not possible to predict by looking at a candidate in isolation.

All of these things are impossible to predict; they’re unique to every team and every form of that team’s composition. They constantly shift as the nature and quality of individual relationships change. And they’re entirely coupled to how we manage and interact with the team and subject to constant outside influence. That sounds far more like a mystery than a puzzle.

The consequences of treating a mystery as a puzzle are uniformly negative. Each new process you go through, each new piece of data you add, is an opportunity to introduce bias rather than a chance to get closer to the solution. Any certainty you have is false, the result only of your having convinced yourself. What seems like additional signal is actually just noise: every additional interview given, every extra test taken, every reference sought. If you hire someone good, it’s less likely to be a result of your exhaustive selection process than a combination of dumb luck and selection bias – the fact that “your vibe attracts your tribe”, that people who fit with your culture are attracted to the public expression of that culture.

What might it look like to treat hiring more like a mystery, to reject false certainty and to embrace uncertainty?

---

In Bolivia, as in much of the world, pupils in secondary school elect a student council: a group of their peers who make decisions about everything from relationships with teachers to the end-of-year dance. And in Bolivia, as in much of the world, the students who run for and win these elections tend to be the most popular, confident, and high-achieving students.

Surprisingly, that process doesn’t select for the best candidates – either the people who could have the greatest impact by serving, or the people upon whom serving could have the greatest impact. It selects for the people self-confident (or self-delusional) enough to campaign, and popular enough to win.

Canadian political scientist Adam Cronkright looked at these elections and saw exactly this problem. He could have approached the problem as a puzzle: sure, there are problems with elections, but with more research and more data we might be able to improve them, find a better process, find some way of determining the best candidate in advance and electing them. But Cronkright realised that this simply isn’t possible: the problem is too complex, and no process can identify in advance who the best candidate might be.

And so, inspired by the direct elections of Athenian democracy, he proposed a radical change: what if, instead of running elections, you appointed the student council via a random lottery instead?

Cronkright set about replacing elections with voluntary lotteries, whereby any student could put their name forward and the final council was selected in a random draw, a draw that ensured that the council reflected the demographics of the student body. And so those candidates who were ostensibly most qualified had exactly the same chance of being elected as those who were ostensibly least qualified.

This goes against our most fundamental instincts: we want so much for there to be a way to select the best people, that the idea of choosing them randomly seems sacrilegious. We want appointing people to the student council to be a puzzle; acknowledging that it’s a mystery, and that randomness is better than false certainty, feels like giving up.

And yet, Cronkright discovered, lotteries actually produce much better outcomes than elections:

“Lotteries change the perception of student government from being rooted in popularity to being rooted in equality. In our work, we found that critical leadership has regularly come from students who struggle with homework, making friends, or speaking in public. They worry us at first, but prove us wrong by showing up for meetings, listening to their teammates, and following through with the things they say they will do.

“Conversely, often our greatest challenge is with students who are at the top of their class, popular, and charismatic. They make great first impressions, but then start showing up late or skipping meetings, try to dominate conversations, and don’t follow through on their commitments. Sometimes they even attempt to hog the credit for their team’s achievements.”

What’s more, Cronkright discovered that practical experience of leadership created more effective leaders overall, by teaching capable students the qualities that they had previously lacked:

“It is also true that confidence and charisma are important aspects of leadership. But especially in democratic settings, leadership requires many other important traits and comes in many diverse forms. We’ve found it much easier to teach respectful and reliable teammates to overcome their fear of public speaking, than it is to teach students who talk the talk to set aside their egos, work well with others, and walk the walk.”

Replacing elections with lotteries acknowledges that the process of elections cannot be fixed, because the problem of identifying the best candidates is not one that can be solved. The outcomes, unsurprisingly, are better than the false certainty of elections.

---

The parallels with hiring are clear. Our obsession with gathering ever-more information and subjecting potential employees to a complicated and convoluted interview process – with treating hiring as a puzzle, in other words – results in selecting for people who can navigate that process. No more, no less. Since those processes generally put people on the spot, ask them to perform in the moment, and are set up as a test that it’s possible to pass or fail, what they select for tends to be confidence, slickness, and charisma – exactly the same as elections.

Just like in elections, you solve that problem by admitting that you can’t have advance knowledge and can’t make an objective decision, and one of the most obvious and effective ways to do that is to embrace randomness.

The idea of introducing randomness into hiring isn’t new. Academic and researcher Amanda Goodall has suggested a process:

“The process would involve three stages. First, human resources will advertise a post and invite strong candidates to apply – not merely relying on the naturally confident. Recruiters may choose to announce at this initial point that the vacant post will, at the final stage, be filled by lot. In the second stage, the promotion panel draws up a list of candidates who meet various performance and other criteria. In stage three, the successful candidate is randomly picked from among this pre-chosen talent pool.”

Just as Cronkright discovered in Bolivia, Goodall argues that such a method wouldn’t just select better people, it would create a better dynamic once those people were selected, since nobody could see their position as the result of anything but randomness:

“Crucially, it also protects women and ethnic minorities against internalising failure, and reduces the propensity of winners to assume they are ‘the chosen one’ and over-claim on their success.”

Likewise, randomness can be used to explicitly guarantee diversity. The behavioural researcher Kent Kroeger has outlined a process of stratified random selection that can ensure that the composition of your teams reflects the demographic makeup of your community – in whatever dimensions you choose.

Our collective efforts to puzzle our way to success have not been successful; they’ve resulted in ineffective, biased, discriminatory hiring practices. Embracing the idea that we just can’t be certain about things is an uncomfortable idea at first, but it’s worth taking a chance on.