Study on patterns and effect of task diversity in software crowdsourcing

Denisse Martinez Mejorado, Razieh Saremi, Ye Yang, Jose E. Ramirez-Marquez

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    5 Scopus citations

    Abstract

    Context: The success of software crowdsourcing depends on steady pools of task demand and active workers supply. Existing analysis reveals an average task failure ratio of 15.7% in software crowdsourcing market. Goal: The objective of this study is to empirically investigate patterns and effect of task diversity in software crowdsourcing platform in order to improve the success and efficiency of software crowdsourcing. Method: We first propose a conceptual task diversity model, and develop an approach to measuring and analyzing task diversity. More specifically, task diversity is characterized based on semantic similarity, dynamic competition level, and the analysis includes identifying the dominant attributes distinguishing the competition levels, and measuring the impact of task diversity on task success and worker performance in crowdsourcing platform. The empirical study is conducted on more than one year's real-world data from TopCoder, one of the leading software crowdsourcing platforms. Results: We identified that monetary prize and task complexity are the dominant attributes that differentiate among different competition levels. Based on these dominant attributes, we concluded three task diversity patterns (configurations) from workers behavior perspective: responsive-to-prize, responsive-to-prize-andcomplexity and over-responsive-to-prize. This study supports that the second pattern, i.e. responsive-to-prize-and-complexity configuration, associates with the lowest task failure ratio. Conclusions: These findings are helpful for task requesters to plan for and improve task success in a more effective and efficient manner in software crowdsourcing platform.

    Original languageEnglish
    Title of host publicationESEM 2020 - Proceedings of the 14th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement
    ISBN (Electronic)9781450375801
    DOIs
    StatePublished - 5 Oct 2020
    Event14th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2020 - Virtual, Online, Italy
    Duration: 5 Oct 20207 Oct 2020

    Publication series

    NameInternational Symposium on Empirical Software Engineering and Measurement
    ISSN (Print)1949-3770
    ISSN (Electronic)1949-3789

    Conference

    Conference14th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2020
    Country/TerritoryItaly
    CityVirtual, Online
    Period5/10/207/10/20

    Keywords

    • Software crowdsourcing
    • Task diversity
    • Task failure
    • Task success
    • Worker performance

    Fingerprint

    Dive into the research topics of 'Study on patterns and effect of task diversity in software crowdsourcing'. Together they form a unique fingerprint.

    Cite this