TY - GEN
T1 - Crowdsourcing more effective initializations for single-target trackers through automatic re-querying
AU - Lemmer, Stephan J.
AU - Song, Jean Y.
AU - Corso, Jason J.
N1 - Publisher Copyright:
© 2021 ACM.
PY - 2021/5/6
Y1 - 2021/5/6
N2 - In single-target video object tracking, an initial bounding box is drawn around a target object and propagated through a video. When this bounding box is provided by a careful human expert, it is expected to yield strong overall tracking performance that can be mimicked at scale by novice crowd workers with the help of advanced quality control methods. However, we show through an investigation of 900 crowdsourced initializations that such quality control strategies are inadequate for this task in two major ways: first, the high level of redundancy in these methods (e.g., averaging multiple responses to reduce error) is unnecessary, as 23% of crowdsourced initializations perform just as well as the gold-standard initialization. Second, even nearly perfect initializations can lead to degraded long-term performance due to the complexity of object tracking. Considering these findings, we evaluate novel approaches for automatically selecting bounding boxes to re-query, and introduce Smart Replacement, an efficient method that decides whether to use the crowdsourced replacement initialization.
AB - In single-target video object tracking, an initial bounding box is drawn around a target object and propagated through a video. When this bounding box is provided by a careful human expert, it is expected to yield strong overall tracking performance that can be mimicked at scale by novice crowd workers with the help of advanced quality control methods. However, we show through an investigation of 900 crowdsourced initializations that such quality control strategies are inadequate for this task in two major ways: first, the high level of redundancy in these methods (e.g., averaging multiple responses to reduce error) is unnecessary, as 23% of crowdsourced initializations perform just as well as the gold-standard initialization. Second, even nearly perfect initializations can lead to degraded long-term performance due to the complexity of object tracking. Considering these findings, we evaluate novel approaches for automatically selecting bounding boxes to re-query, and introduce Smart Replacement, an efficient method that decides whether to use the crowdsourced replacement initialization.
KW - Crowd-ai collaboration
KW - Crowdsourcing
KW - Seed rejection
KW - Single-target video object tracking
KW - Smart replacement
UR - http://www.scopus.com/inward/record.url?scp=85106751575&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85106751575&partnerID=8YFLogxK
U2 - 10.1145/3411764.3445181
DO - 10.1145/3411764.3445181
M3 - Conference contribution
AN - SCOPUS:85106751575
T3 - Conference on Human Factors in Computing Systems - Proceedings
BT - CHI 2021 - Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
T2 - 2021 CHI Conference on Human Factors in Computing Systems: Making Waves, Combining Strengths, CHI 2021
Y2 - 8 May 2021 through 13 May 2021
ER -