Towards kilo-hertz 6-DoF visual tracking using an egocentric cluster of rolling shutter cameras

Akash Bapat, Enrique Dunn, Jan Michael Frahm

Research output: Contribution to journalArticlepeer-review

14 Scopus citations

Abstract

To maintain a reliable registration of the virtual world with the real world, augmented reality (AR) applications require highly accurate, low-latency tracking of the device. In this paper, we propose a novel method for performing this fast 6-DOF head pose tracking using a cluster of rolling shutter cameras. The key idea is that a rolling shutter camera works by capturing the rows of an image in rapid succession, essentially acting as a high-frequency 1D image sensor. By integrating multiple rolling shutter cameras on the AR device, our tracker is able to perform 6-DOF markerless tracking in a static indoor environment with minimal latency. Compared to state-of-the-art tracking systems, this tracking approach performs at significantly higher frequency, and it works in generalized environments. To demonstrate the feasibility of our system, we present thorough evaluations on synthetically generated data with tracking frequencies reaching 56.7 kHz. We further validate the method's accuracy on real-world images collected from a prototype of our tracking system against ground truth data using standard commodity GoPro cameras capturing at 120 Hz frame rate.

Original languageEnglish
Article number7523411
Pages (from-to)2358-2367
Number of pages10
JournalIEEE Transactions on Visualization and Computer Graphics
Volume22
Issue number11
DOIs
StatePublished - Nov 2016

Keywords

  • High frequency
  • Rolling shutter
  • Visual inside-out tracking

Fingerprint

Dive into the research topics of 'Towards kilo-hertz 6-DoF visual tracking using an egocentric cluster of rolling shutter cameras'. Together they form a unique fingerprint.

Cite this