TY - GEN
T1 - Pairwise Ranking Aggregation by Non-interactive Crowdsourcing with Budget Constraints
AU - Cai, Changjiang
AU - Sun, Haipei
AU - Dong, Boxiang
AU - Zhang, Bo
AU - Wang, Ting
AU - Wang, Hui
N1 - Publisher Copyright:
© 2017 IEEE.
PY - 2017/7/13
Y1 - 2017/7/13
N2 - Crowdsourced ranking algorithms ask the crowd to compare the objects and infer the full ranking based on the crowdsourced pairwise comparison results. In this paper, we consider the setting in which the task requester is equipped with a limited budget that can afford only a small number of pairwise comparisons. To make the problem more complicated, the crowd may return noisy comparison answers. We propose an approach to obtain a good-quality full ranking from a small number of pairwise preferences in two steps, namely task assignment and result inference. In the task assignment step, we generate pairwise comparison tasks that produce a full ranking with high probability. In the result inference step, based on the transitive property of pairwise comparisons and truth discovery, we design an efficient heuristic algorithm to find the best full ranking from the potentially conflictive pairwise preferences. The experiment results demonstrate the effectiveness and efficiency of our approach.
AB - Crowdsourced ranking algorithms ask the crowd to compare the objects and infer the full ranking based on the crowdsourced pairwise comparison results. In this paper, we consider the setting in which the task requester is equipped with a limited budget that can afford only a small number of pairwise comparisons. To make the problem more complicated, the crowd may return noisy comparison answers. We propose an approach to obtain a good-quality full ranking from a small number of pairwise preferences in two steps, namely task assignment and result inference. In the task assignment step, we generate pairwise comparison tasks that produce a full ranking with high probability. In the result inference step, based on the transitive property of pairwise comparisons and truth discovery, we design an efficient heuristic algorithm to find the best full ranking from the potentially conflictive pairwise preferences. The experiment results demonstrate the effectiveness and efficiency of our approach.
KW - Budget constraint
KW - Crowdsourcing
KW - Rank aggregation
UR - http://www.scopus.com/inward/record.url?scp=85027256394&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85027256394&partnerID=8YFLogxK
U2 - 10.1109/ICDCS.2017.102
DO - 10.1109/ICDCS.2017.102
M3 - Conference contribution
AN - SCOPUS:85027256394
T3 - Proceedings - International Conference on Distributed Computing Systems
SP - 2567
EP - 2568
BT - Proceedings - IEEE 37th International Conference on Distributed Computing Systems, ICDCS 2017
A2 - Lee, Kisung
A2 - Liu, Ling
T2 - 37th IEEE International Conference on Distributed Computing Systems, ICDCS 2017
Y2 - 5 June 2017 through 8 June 2017
ER -