Depth sensing and augmented reality technologies for mobile 3d platforms

This paper presents novel depth sensing and augmented reality technologies based on mobile 3D platforms. The mobile stereoscopic cameras are utilized to provide a “soft depth sensor” generating instant depth estimation of the physical world. The depth sensing algorithms are proposed to recover the 3D geometric structure captured in single or multiple stereoscopic images. Then, two augmented reality applications are developed to showcase the advantages of the depth sensing capability, namely visual measurement and a 3D AR shooter game. In particular, the AR technology allows occlusion between physical and virtual objects. Both visual and quantitative results are shown to illustrate the novelty and effectiveness of the proposed algorithms.