Spotlight on Recruiting Tech: What Works — and What Doesn't — with Game-Based Evaluation

Is the process of hiring and promoting the right people an art — or a science? The debate is raging in human resources circles with the rise of new technologies that claim to quickly and accurately assess whether a worker is the best fit for a job.

How do these technologies do it? Through "predictive people analytics," which are computer-based assessment tools that use algorithms to calculate a job candidate's potential. Proponents say people analytics ensure that companies make better hiring and promotion decisions. Critics, however, contend that no technology can replicate the value of human evaluation. Maybe technology can assess if someone has the right skills, they concede. But what about softer skills like integrity or personality? No way.

Here's a closer look at debate through Knack, an up-and-coming people analytics service. Knack develops app-based video games that are intended to calculate a worker's potential as a leader or as an innovator based on how she performs in a series of games. "Dungeon Scrawl," for instance, challenges an employee to find her way through a maze and solve puzzles. "Wasabi Waiter" turns a user into a waiter — and tests her ability to bring the right sushi order to the right customers as the restaurant gets busier.

Guy Halfteck, the founder of Knack, tells The Atlantic that Knack games generate far more data in a much shorter period of time than any SAT or standard personality test could. The games not only analyze whether workers can complete tasks, but also measure how long they hesitate before making a move and the paths users take to complete a task. All told, says Halfteck, Knack measures six traits — or knacks — to determine if users are likely to be innovators:

In the end, Halfteck tells The Atlantic, you get a "high-resolution portrait of [an applicant's] psyche and intellect, and an assessment of [her] potential as a leader or an innovator."

Royal Dutch Shell, the oil giant, is one company that uses Knack to assess for a skill that's especially hard to measure: creativity and innovation. Shell's GameChanger division employs inventors who can spot disruptive ideas — whether they come from within the company or from outside. Knack helps Shell identify workers who can identify and evaluate business ideas, only 10 percent of which actually get pursued to any degree, Hans Haringa of Shell tells The Atlantic.

Haringa says he was skeptical at first. To test Knack’s algorithm, he asked individuals who previously contributed an idea to GameChanger to play Knack’s games. The result: the same individuals whose ideas Shell had singled out were identified by Knack as innovators — only in far less time. Today, Shell's human resources team uses Knack for recruiting. Haringa calls Knack a "paradigm shift."

Catherine Rampbell at The New York Times sees the potential for people analytics to new open doors for job seekers — say, those without a college degree. But she questions whether today's technology can "replace the good old-fashioned grill session" just yet. People analytics, she argues, cannot assess soft skills such as empathy or honesty.

John Sumser, the editor-in-chief of HRExaminer Online Magazine, is equally doubtful. Any good recruiter can quickly assess a candidate's skills, or lack thereof, by looking at a resume. He doubts, too, that anyone would want a "doctor or lawyer recruited by a game." That said, Sumser sees value in Knack's technology for employers who need to fill a lot of low-skilled jobs — such as a restaurant manager who's on the constant lookout for waiters.

A better alternative to people analytics, according to Melissa Hooven of Cornerstone OnDemand, is behavioral interviewing, such as the STAR method. With STAR, interviewers ask candidates to describe a situation, task or action — and the results ("STAR") achieved when answering each question. "If it’s clear that an interview is always answering with incomplete STARs, it’s clear that he’s not going to be a good candidate — or employee," argues Hooven.

What's the verdict on Knack? It's too soon to tell. But the consensus so far is that the process of hiring and promoting is still very much an art form. "Nothing in the science of prediction and selection beats observing actual performance in an equivalent role," Peter Cappelli, a professor at the Wharton School, tells The Atlantic.

Photo credit: Knack