In 2006, Netflix decided to hold an open contest to help improve its recommendation algorithm, Cinematch. Armed with user data, contestants competed to design an algorithm that could outperform Cinematch in predicting how users would rate a new movie based on their previous preferences. It took nearly three years, but finally a pair of teams succeeded in beating Cinematch by at least 10 percent—the criteria for ending the contest and declaring a winner.
In recent decades, innovation challenges, hackathons, and other open-innovation contests have become quite popular, particularly in tech. But the broader use of contests as a powerful, cost-effective way of rewarding effort spans all industries. U.S. governmental agencies hold prize competitions that spur the public to solve problems. Nonprofits regularly hold fundraising contests that push donors to compete for prizes or recognition. Managers hold contests—though they might not explicitly describe them as such—to encourage employees to compete for a coveted promotion. And in the far-flung world of cryptocurrencies, Bitcoin miners compete against one another to win Bitcoin by being the first to verify transactions in the blockchain.
“There are a lot of different settings where we see contests being used in practice,” says George Georgiadis, an associate professor of strategy at Kellogg.
Still, despite their ubiquity, holding an optimal contest—that is, one that gets the most bang for its buck in terms of motivating competitors to reach a desired goal—is surprisingly tricky. After all, the contest designer has a number of important decisions to make. When exactly should the contest end? How should the prize be allocated? And what is the best strategy for providing contestants with feedback about their own performance and that of their competitors?
New research by Georgiadis and colleagues Luis Rayo, a professor of strategy at Kellogg; Jeffrey Ely, a professor of economics at Northwestern (and a professor of managerial economics and decision sciences by courtesy at Kellogg); and Sina Khorasani, a postdoc at University of California San Diego, offers answers.
The researchers conclude that, in many common scenarios, the deadline for finishing the contest should be provisional, meaning that if nobody reaches the end-goal, the deadline is extended. The researchers also find that in such scenarios, the optimal contest splits the prize equally between all competitors who succeed before the deadline and provides competitors with limited, but strategic, feedback.
“Our goal is trying to come up with the best possible contest under certain conditions,” says Rayo. “Maybe our results help us understand some features of real-life contests, but they also suggest improvements.”
Considering a Contest
Organizations regularly use incentives to motivate individuals and teams to work hard. Often, this takes the shape of a contract that dictates how an employee will be compensated: they might receive a base wage, for instance, but also have the potential to earn a bonus if they reach a certain performance milestone. Or perhaps their pay might increase linearly as their performance increases.
Researchers have long studied how to design the optimal contract under different situations. But there are times where it may make sense for organizations to use a contest instead.
“Which is better? It depends,” says Rayo. “One of the things we show in this paper is a situation where it is actually better to have employees compete against one another.”
Specifically, the researchers demonstrate that when employers are able to control the flow of information and strategically withhold information about how rival competitors are doing, organizations can potentially get more bang for their buck by offering a contest than by drawing up individual incentive contracts.
“When you’re able to control that flow of information, that’s an important reason why you may want to have a tournament rather than rewarding each agent independently of how others do,” says Rayo.
The Optimal Contest
To better understand when contests are most useful—and how to design an optimal one—the researchers built a mathematical model that describes how employees will work under various compensation schemes. In their model, employers want employees to work hard for as long as possible. In the process, they might reach a certain goal, such as building an algorithm that is 10 percent more effective, hitting a certain sales target, or becoming promotion-worthy. As soon as the goal is accomplished by one of the contestants, the employer is notified.
The researchers then tweaked several aspects of the contest in order to determine which design would motivate contestants to work as hard as possible for the same amount of prize money.
“We’re trying to stretch this prize money as much as possible,” says Rayo.
One component they varied was how the employer shared information with contestants. What would happen if the employer informed all of the competitors as soon as a single contestant accomplished the goal? Alternatively, the employer could regularly inform the competitors that nobody had yet succeeded. Or the employer could decide to share information randomly, so that sometimes competitors would be told about others’ performance, and other times they would not.
The researchers also considered how the contest should end. Here, too, there were several possibilities. The contest could end as soon as one competitor accomplished the desired goal, or all of the competitors could be given a hard deadline, after which the competition would end, even if nobody had yet finished. Or competitors could be given a provisional deadline, but the contest could be extended.
Finally, the researchers varied how the prize was allocated: Should it be winner-takes-all, assigned randomly, or split equally among all of the competitors who succeeded?
They found the optimal contest used a provisional deadline. Contestants heard nothing about others’ performance until that deadline. However, if the provisional deadline passed and a new one was set, the contestants could then infer that nobody had yet succeeded. This would continue until one or more contestants did succeed, at which point the contest would end at the next provisional deadline, and the prize would be shared equally amongst every successful competitor.
Why is this the optimal contest design?
For as long a time as possible, even after a competitor finishes, the employer wants to stay mum so as not to discourage those who are still working. So keeping the contest going (and keeping quiet) is their best bet.
But after a certain amount of time, competitors are likely to get dejected and lose the will to compete. So the ability to extend a provisional deadline allows the employer to offer some much-needed motivation by signaling that “it’s safe to keep going. Nobody has yet succeeded,” says Rayo.
And dividing the prize equally among everyone who succeeds is ideal, because if it were to go solely to the competitor who finishes first, contestants might lose motivation over time, assuming that even if they did finish before the deadline, they would be unlikely to finish first. “It’s important to not discourage people from working toward the end of the cycle,” says Rayo.
Georgiadis points out that all the various design components complement one another to produce an optimal contest. If the provisional deadlines are set too close together, for instance, then the equal-allocation rule for the prize money is highly motivating, because competitors will assume that not very many of them will achieve the goal before deadline, so there will be larger shares of prize money doled out. But the contest will end prematurely, leaving effort on the table. On the other hand, if the provisional deadlines are set too far apart (or dropped altogether), then the equal-allocation rule is not very motivating, because competitors will assume that the prize will be split among too many competitors.
“The feedback policy goes hand in hand with the allocation rule,” says Georgiadis.
In the Real World
The research points to ways that existing contests can potentially be improved. Take Bitcoin, for instance, which uses a competition to motivate miners to verify updates to the blockchain. “What we show is that if you replace the current system with what we proposed, you can get the same amount of effort while having to issue less Bitcoin,” says Georgiadis.
More broadly, the researchers were surprised by just how cost-effective this type of contest can be. “As a firm, what you want to do is create a lot of value and capture as much of it as possible,” says Georgiadis. “What this contest does is it creates the maximum value it can possibly create given the prize money, and it manages to capture all of it.”
This is what makes contests superior to individual incentive contracts—at least in situations where an employer maintains control over the flow of information.
“If you were to contract with employees one by one, that means with some probability, you would be wasting the prize,” says Georgiadis. But the contest format essentially allows the same prize money that would have otherwise been wasted on Contestant 1 to motivate Contestant 2. Contestants’ uncertainty about their fellow competitors’ success—and thus whether it is truly worth their while to continue working—is what allows the organization to extract more mileage from the prize.
Another way of thinking about it, however, is that if employers are capturing all the value, the employees are capturing none it. “The employees are not thrilled about this,” says Rayo.
He points out that many organizations have other goals—such building morale or loyalty. In that case, deliberately designing a less-efficient contest could be the ticket.
“If you wanted the employees to keep some value to themselves, you would make these deadlines closer to each other, instead of being silent for, say, three months, and then revealing that nobody has succeeded,” says Rayo. This would provide competitors with much better information about their odds of winning a large share of the prize, helping them to make informed decisions about whether to continue working.
“You’re keeping them less in the dark,” says Rayo.
This article originally appeared in Kellogg Insight, a publication of the Kellogg School of Management at Northwestern University. It is used with permission.