Joint occlusion boundary detection and figure/ground assignment by extracting common-fate fragments in a back-projection scheme

Occlusion boundary detection and figure/ground assignment are among the fundamental challenges for the real world visual pattern recognition applications, such as 3D spatial understanding, robotic navigation and object search. We attack these challenges by extracting an intermediate-level image/video representation, namely, Common-Fate Fragments. A Common-Fate Fragment is composed of both over-segmented region and edge fragments. Physically, it exists as a coupled edge-region fragment bound with dynamic information. Common-Fate Fragment candidates are generated by an integrated line-region growing process, which does not require complete object segmentation or closed object boundary extraction. To identify Common-Fate Fragments from these extracted candidates, we introduce a back-projection verification scheme that can circumvent the notoriously difficult task of direct motion estimation on boundaries. This allows occlusion detection and figure/ground labeling to be jointly conducted within a simple but effective hypothesize-and-test framework. We test the proposed method on YouTube Motion Boundaries (YMB) data set and two benchmark data sets: the CMU and Berkeley motion data sets. Even though the idea of the proposed method is simple and transparent, promising experimental results are observed.