Teams vs. Crowds: A Field Test of the Relative Contribution of Incentives, Member Ability, and Collaboration to Crowd-Based Problem Solving Performance

Organizations are increasingly turning to crowdsourcing to solve difficult problems. This is often driven by the desire to find the best subject matter experts, strongly incentivize them, and engage them with as little coordination cost as possible. A growing number of authors, however, are calling for increased collaboration in crowdsourcing settings, hoping to draw upon the advantages of teamwork observed in traditional settings. The question is how to effectively incorporate team-based collaboration in a setting that has traditionally been individual-based. We report on a large-field experiment of team collaboration on an online platform, in which incentives and team membership were randomly assigned, to evaluate the influence of exogenous inputs (member skills and incentives) and emergent collaboration processes on performance of crowd-based teams. Building on advances in machine learning and complex systems theory, we leverage new measurement techniques to examine the content and timing of team collaboration. We find that temporal "burstiness" of team activity and the diversity of information exchanged among team members are strong predictors of performance, even when inputs such as incentives and member skills are controlled. We discuss implications for research on crowdsourcing and team collaboration. This well-written paper focuses on the phenomenon of crowdsourcing and asks the question: How might groups of individuals collaborate most effectively in a crowdsourcing setting to produce high quality solutions to problems? The paper describes a rigorous field study with random assignment of individuals to groups that seeks to examine the conditions that could facilitate a team's performance on a problem-solving task in a crowd-based setting. The "discovery" is that the temporal "burstiness" of the team members' contributions, which suggests some effort to coordinate attention to the problem, plays a highly significant role in influencing the quality of solutions that teams produce. As one of the reviewers noted - this paper is a perfect "fit" for the Academy of Management Discoveries. I wholeheartedly agree it focuses on an important yet poorly understood phenomenon and reports on the results of a rigorous field study that provides potentially important insights into developing our understanding of that phenomenon. I highly recommend that all Academy members read this paper.