GENIE TRECVID2011 multimedia event detection: Late-fusion approaches to combine multiple audio-visual features

Amitha A.G. Perera; Sangmin Oh; Matthew Leotta; Ilseo Kim; Byuungki Byun; Chin Hui Lee; Scott McCloskey; Jingchen Liu; Ben Miller; Zhi Feng Huang; Arash Vahdat; Weilong Yang; Greg Mori; Kevin Tang; Daphne Koller; L. Fei-Fei; Kang Li; Gang Chen; Jason Corso; Yun Fu; Rohini Srihari

GENIE TRECVID2011 multimedia event detection: Late-fusion approaches to combine multiple audio-visual features

Amitha A.G. Perera
, Sangmin Oh
, Matthew Leotta
, Ilseo Kim
, Byuungki Byun
, Chin Hui Lee
, Scott McCloskey
, Jingchen Liu
, Ben Miller
, Zhi Feng Huang
, Arash Vahdat
, Weilong Yang
, Greg Mori
, Kevin Tang
, Daphne Koller
, L. Fei-Fei
, Kang Li
, Gang Chen
, Jason Corso
, Yun Fu

Rohini Srihari

Department of Computer Science

Research output: Contribution to conference › Paper › peer-review

8 Scopus citations

Abstract

For TRECVID 2011 MED task, the GENIE system incorporated two late-fusion approaches where multiple discriminative base-classifiers are built per feature, then, combined later through discriminative fusion techniques. All of our fusion and base classifiers are formulated as one-vs-all detectors per event class along with threshold estimation capabilities during cross-validation. Total of five different types of features were extracted from data, which include both audio or visual features: HOG3D, Object Bank, Gist, MFCC, and acoustic segment models (ASMs). Features such as HOG3D and MFCC are low-level features while Object Bank and ASMs are more semantic. In our work, event-specific feature adaptations or manual annotations were deliberately avoided, to establish a strong baseline results. Overall, the results were competitive in the MED11 evaluation, and shows that standard machine learning techniques can yield fairly good results even on a challenging dataset.

Original language	English
State	Published - 2011
Event	TREC Video Retrieval Evaluation, TRECVID 2011 - Gaithersburg, MD, United States Duration: 5 Dec 2011 → 7 Dec 2011

Conference

Conference	TREC Video Retrieval Evaluation, TRECVID 2011
Country/Territory	United States
City	Gaithersburg, MD
Period	5/12/11 → 7/12/11

Cite this

Perera, A. A. G., Oh, S., Leotta, M., Kim, I., Byun, B., Lee, C. H., McCloskey, S., Liu, J., Miller, B., Huang, Z. F., Vahdat, A., Yang, W., Mori, G., Tang, K., Koller, D., Fei-Fei, L., Li, K., Chen, G., Corso, J., ... Srihari, R. (2011). GENIE TRECVID2011 multimedia event detection: Late-fusion approaches to combine multiple audio-visual features. Paper presented at TREC Video Retrieval Evaluation, TRECVID 2011, Gaithersburg, MD, United States.

@conference{580bb0f7c1df4b4a8da6ed3e416832af,

title = "GENIE TRECVID2011 multimedia event detection: Late-fusion approaches to combine multiple audio-visual features",

abstract = "For TRECVID 2011 MED task, the GENIE system incorporated two late-fusion approaches where multiple discriminative base-classifiers are built per feature, then, combined later through discriminative fusion techniques. All of our fusion and base classifiers are formulated as one-vs-all detectors per event class along with threshold estimation capabilities during cross-validation. Total of five different types of features were extracted from data, which include both audio or visual features: HOG3D, Object Bank, Gist, MFCC, and acoustic segment models (ASMs). Features such as HOG3D and MFCC are low-level features while Object Bank and ASMs are more semantic. In our work, event-specific feature adaptations or manual annotations were deliberately avoided, to establish a strong baseline results. Overall, the results were competitive in the MED11 evaluation, and shows that standard machine learning techniques can yield fairly good results even on a challenging dataset.",

author = "Perera, \{Amitha A.G.\} and Sangmin Oh and Matthew Leotta and Ilseo Kim and Byuungki Byun and Lee, \{Chin Hui\} and Scott McCloskey and Jingchen Liu and Ben Miller and Huang, \{Zhi Feng\} and Arash Vahdat and Weilong Yang and Greg Mori and Kevin Tang and Daphne Koller and L. Fei-Fei and Kang Li and Gang Chen and Jason Corso and Yun Fu and Rohini Srihari",

year = "2011",

language = "English",

note = "TREC Video Retrieval Evaluation, TRECVID 2011 ; Conference date: 05-12-2011 Through 07-12-2011",

}

Perera, AAG, Oh, S, Leotta, M, Kim, I, Byun, B, Lee, CH, McCloskey, S, Liu, J, Miller, B, Huang, ZF, Vahdat, A, Yang, W, Mori, G, Tang, K, Koller, D, Fei-Fei, L, Li, K, Chen, G, Corso, J, Fu, Y & Srihari, R 2011, 'GENIE TRECVID2011 multimedia event detection: Late-fusion approaches to combine multiple audio-visual features', Paper presented at TREC Video Retrieval Evaluation, TRECVID 2011, Gaithersburg, MD, United States, 5/12/11 - 7/12/11.

TY - CONF

T1 - GENIE TRECVID2011 multimedia event detection

T2 - TREC Video Retrieval Evaluation, TRECVID 2011

AU - Perera, Amitha A.G.

AU - Oh, Sangmin

AU - Leotta, Matthew

AU - Kim, Ilseo

AU - Byun, Byuungki

AU - Lee, Chin Hui

AU - McCloskey, Scott

AU - Liu, Jingchen

AU - Miller, Ben

AU - Huang, Zhi Feng

AU - Vahdat, Arash

AU - Yang, Weilong

AU - Mori, Greg

AU - Tang, Kevin

AU - Koller, Daphne

AU - Fei-Fei, L.

AU - Li, Kang

AU - Chen, Gang

AU - Corso, Jason

AU - Fu, Yun

AU - Srihari, Rohini

PY - 2011

Y1 - 2011

N2 - For TRECVID 2011 MED task, the GENIE system incorporated two late-fusion approaches where multiple discriminative base-classifiers are built per feature, then, combined later through discriminative fusion techniques. All of our fusion and base classifiers are formulated as one-vs-all detectors per event class along with threshold estimation capabilities during cross-validation. Total of five different types of features were extracted from data, which include both audio or visual features: HOG3D, Object Bank, Gist, MFCC, and acoustic segment models (ASMs). Features such as HOG3D and MFCC are low-level features while Object Bank and ASMs are more semantic. In our work, event-specific feature adaptations or manual annotations were deliberately avoided, to establish a strong baseline results. Overall, the results were competitive in the MED11 evaluation, and shows that standard machine learning techniques can yield fairly good results even on a challenging dataset.

AB - For TRECVID 2011 MED task, the GENIE system incorporated two late-fusion approaches where multiple discriminative base-classifiers are built per feature, then, combined later through discriminative fusion techniques. All of our fusion and base classifiers are formulated as one-vs-all detectors per event class along with threshold estimation capabilities during cross-validation. Total of five different types of features were extracted from data, which include both audio or visual features: HOG3D, Object Bank, Gist, MFCC, and acoustic segment models (ASMs). Features such as HOG3D and MFCC are low-level features while Object Bank and ASMs are more semantic. In our work, event-specific feature adaptations or manual annotations were deliberately avoided, to establish a strong baseline results. Overall, the results were competitive in the MED11 evaluation, and shows that standard machine learning techniques can yield fairly good results even on a challenging dataset.

UR - https://www.scopus.com/pages/publications/84905269591

UR - https://www.scopus.com/pages/publications/84905269591#tab=citedBy

M3 - Paper

AN - SCOPUS:84905269591

Y2 - 5 December 2011 through 7 December 2011

ER -

GENIE TRECVID2011 multimedia event detection: Late-fusion approaches to combine multiple audio-visual features

Abstract

Conference

Other files and links

Fingerprint

Cite this