Study on patterns and effect of task diversity in software crowdsourcing

Context: The success of software crowdsourcing depends on steady pools of task demand and active workers supply. Existing analysis reveals an average task failure ratio of 15.7% in software crowdsourcing market. Goal: The objective of this study is to empirically investigate patterns and effect of task diversity in software crowdsourcing platform in order to improve the success and efficiency of software crowdsourcing. Method: We first propose a conceptual task diversity model, and develop an approach to measuring and analyzing task diversity. More specifically, task diversity is characterized based on semantic similarity, dynamic competition level, and the analysis includes identifying the dominant attributes distinguishing the competition levels, and measuring the impact of task diversity on task success and worker performance in crowdsourcing platform. The empirical study is conducted on more than one year's real-world data from TopCoder, one of the leading software crowdsourcing platforms. Results: We identified that monetary prize and task complexity are the dominant attributes that differentiate among different competition levels. Based on these dominant attributes, we concluded three task diversity patterns (configurations) from workers behavior perspective: responsive-to-prize, responsive-to-prize-andcomplexity and over-responsive-to-prize. This study supports that the second pattern, i.e. responsive-to-prize-and-complexity configuration, associates with the lowest task failure ratio. Conclusions: These findings are helpful for task requesters to plan for and improve task success in a more effective and efficient manner in software crowdsourcing platform.