Temporally consistent reconstruction from multiple video streams using enhanced belief propagation

E. Scott Larsen, Philippos Mordohai, Marc Pollefeys, Henry Fuchs

Research output: Contribution to conferencePaperpeer-review

78 Scopus citations

Abstract

We present an approach for 3D reconstruction from multiple video streams taken by static, synchronized and calibrated cameras that is capable of enforcing temporal consistency on the reconstruction of successive frames. Our goal is to improve the quality of the reconstruction by finding corresponding pixels in subsequent frames of the same camera using optical flow, and also to at least maintain the quality of the single time-frame reconstruction when these correspondences are wrong or cannot be found. This allows us to process scenes with fast motion, occlusions and self-occlusions where optical flow fails for large numbers of pixels. To this end, we modify the belief propagation algorithm to operate on a 3D graph that includes both spatial and temporal neighbors and to be able to discard messages from outlying neighbors. We also propose methods for introducing a bias and for suppressing noise typically observed in uniform regions. The bias encapsulates information about the background and aids in achieving a temporally consistent reconstruction and in the mitigation of occlusion related errors. We present results on publicly available real video sequences. We also present quantitative comparisons with results obtained by other researchers.

Original languageEnglish
DOIs
StatePublished - 2007
Event2007 IEEE 11th International Conference on Computer Vision, ICCV - Rio de Janeiro, Brazil
Duration: 14 Oct 200721 Oct 2007

Conference

Conference2007 IEEE 11th International Conference on Computer Vision, ICCV
Country/TerritoryBrazil
CityRio de Janeiro
Period14/10/0721/10/07

Fingerprint

Dive into the research topics of 'Temporally consistent reconstruction from multiple video streams using enhanced belief propagation'. Together they form a unique fingerprint.

Cite this