PopUp: Reconstructing 3D video using particle filtering to aggregate crowd responses

Jean Y. Song, Stephan J. Lemmer, Michael Xieyang Liu, Shiyan Yan, Juho Kim, Jason J. Corso, Walter S. Lasecki

Research output: Contribution to conferencePaperpeer-review

13 Scopus citations

Abstract

Collecting a sufficient amount of 3D training data for autonomous vehicles to handle rare, but critical, traffic events (e.g., collisions) may take decades of deployment. Abundant video data of such events from municipal traffic cameras and video sharing sites (e.g., YouTube) could provide a potential alternative, but generating realistic training data in the form of 3D video reconstructions is a challenging task beyond the current capabilities of computer vision. Crowdsourcing the annotation of necessary information could bridge this gap, but the level of accuracy required to obtain usable reconstructions makes this task nearly impossible for non-experts. In this paper, we propose a novel hybrid intelligence method that combines annotations from workers viewing different instances (video frames) of the same target (3D object), and uses particle filtering to aggregate responses. Our approach can leveraging temporal dependencies between video frames, enabling higher quality through more aggressive filtering. The proposed method results in a 33% reduction in the relative error of position estimation compared to a state-of-the-art baseline. Moreover, our method enables skipping (self-filtering) challenging annotations, reducing the total annotation time for hard-to-annotate frames by 16%. Our approach provides a generalizable means of aggregating more accurate crowd responses in settings where annotation is especially challenging or error-prone.

Original languageEnglish
Pages558-569
Number of pages12
DOIs
StatePublished - 2019
Event24th ACM International Conference on Intelligent User Interfaces, IUI 2019 - Marina del Ray, United States
Duration: 17 Mar 201920 Mar 2019

Conference

Conference24th ACM International Conference on Intelligent User Interfaces, IUI 2019
Country/TerritoryUnited States
CityMarina del Ray
Period17/03/1920/03/19

Keywords

  • 3D Reconstruction
  • Answer Aggregation
  • Autonomous Vehicle
  • Crowdsourcing
  • Human Computation
  • Particle Filter

Fingerprint

Dive into the research topics of 'PopUp: Reconstructing 3D video using particle filtering to aggregate crowd responses'. Together they form a unique fingerprint.

Cite this