Timely Quality Problem Resolution in Peer-Production Systems: The Impact of Bots, Policy Citations, and Contributor Experience

Although online peer-production systems have proven to be effective in producing high-quality content, their open call for participation makes them susceptible to ongoing quality problems. A key concern is that the problems should be addressed quickly to prevent low-quality content from remaining in place for extended periods. We examine the impacts of two control mechanisms, bots and policy citations, and the number of contributors, with and without prior experience in editing an article, on the cleanup time of 4,473 quality problem events in Wikipedia. We define cleanup time as the time it takes to resolve a quality problem once it has been detected in an article. Using an accelerated failure time model, we find that the number of bots editing an article during a quality problem event has no effect on cleanup time; that citing policies to justify edits during the event is associated with a longer cleanup time; and that more contributors, with or without prior experience in editing the article, are associated with a shorter cleanup time. We also find important interactions between each of the two control mechanisms and the number of different types of contributors. There is a marginal increase in cleanup time that is larger when an increase in the number of contributors is accompanied by fewer bots editing the article during a quality problem event. This interaction effect is more pronounced when increasing the number of contributors without prior experience in editing the article. Further, there is a marginal decrease in cleanup time that is larger when an increase in the number of contributors, with or without prior experience in editing the article, is accompanied by fewer policy citations. Taken together, our results show that the use of bots and policy citations as control mechanisms must be considered in conjunction with the number of contributors with and without prior experience in editing an article. Accordingly, the number of contributors and their experience alone may not explain important outcomes in peer production; it is also important to find an appropriate mix of different control mechanisms and types of contributors to address quality problems quickly.