TY - GEN
T1 - A new optical flow estimation method in joint EO/IR video surveillance
AU - Man, Hong
AU - Holt, Robert J.
AU - Wang, Jing
AU - Martini, Rainer
AU - Netravali, Ravi
AU - Mukherjee, Iraban
PY - 2007
Y1 - 2007
N2 - Electro-Optical (EO) and Infra-Red (IR) sensors have been jointly deployed in many surveillance systems. In this work we study the special characteristics of optical flow in IR imagery, and introduce an optical flow estimation method using co-registered EO and IR image frames. The basic optical flow calculation is based on the combined local and global (CLG) method (Bruhn, Weickert and Schnorr, 2002), which seeks solutions that simultaneously satisfy a local averaged brightness constancy constraint and a global flow smoothness constraint. While CLG method can be directly applied to IR image frames, the estimated optical flow fields usually manifest high level of random motions caused by thermal noise. Furthermore, IR sensors operating at different wavelengths, e.g. meddle-wave infrared (MWIR) and long-wave infrared (LWIR), may yield inconsistent motions in optical flow estimation. Because of the availability of both EO and IR sensors in many practical scenarios, we propose to estimate optical flow jointly using both EO and IR image frames. This method is able to take advantage of the complementary information offered by these two imaging modalities. The joint optical flow calculation fuses the motion fields from EO and IR images using a cross-regularization mechanism and a non-linear flow fusion model which aligns the estimated motions based on neighbor activities. Experiments performed on the OTCBVS dataset demonstrated that the proposed approach can effectively eliminate many unimportant motions, and significantly reduce erroneous motions, such as sensor noise.
AB - Electro-Optical (EO) and Infra-Red (IR) sensors have been jointly deployed in many surveillance systems. In this work we study the special characteristics of optical flow in IR imagery, and introduce an optical flow estimation method using co-registered EO and IR image frames. The basic optical flow calculation is based on the combined local and global (CLG) method (Bruhn, Weickert and Schnorr, 2002), which seeks solutions that simultaneously satisfy a local averaged brightness constancy constraint and a global flow smoothness constraint. While CLG method can be directly applied to IR image frames, the estimated optical flow fields usually manifest high level of random motions caused by thermal noise. Furthermore, IR sensors operating at different wavelengths, e.g. meddle-wave infrared (MWIR) and long-wave infrared (LWIR), may yield inconsistent motions in optical flow estimation. Because of the availability of both EO and IR sensors in many practical scenarios, we propose to estimate optical flow jointly using both EO and IR image frames. This method is able to take advantage of the complementary information offered by these two imaging modalities. The joint optical flow calculation fuses the motion fields from EO and IR images using a cross-regularization mechanism and a non-linear flow fusion model which aligns the estimated motions based on neighbor activities. Experiments performed on the OTCBVS dataset demonstrated that the proposed approach can effectively eliminate many unimportant motions, and significantly reduce erroneous motions, such as sensor noise.
KW - Infrared imaging
KW - Multi-sensor fusion
KW - Multimodal imagery
KW - Optical flow
KW - Visual surveillance
UR - http://www.scopus.com/inward/record.url?scp=35948967982&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=35948967982&partnerID=8YFLogxK
U2 - 10.1117/12.719164
DO - 10.1117/12.719164
M3 - Conference contribution
AN - SCOPUS:35948967982
SN - 0819466654
SN - 9780819466655
T3 - Proceedings of SPIE - The International Society for Optical Engineering
BT - Infrared Imaging Systems
T2 - Infrared Imaging Systems: Design, Analysis, Modeling, and Testing XVIII
Y2 - 11 April 2007 through 13 April 2007
ER -