TY - GEN
T1 - Systemic test and evaluation of a hard+soft information fusion framework
T2 - 17th International Conference on Information Fusion, FUSION 2014
AU - Gross, Geoff A.
AU - Date, Ketan
AU - Schlegel, Daniel R.
AU - Corso, Jason J.
AU - Llinas, James
AU - Nagi, Rakesh
AU - Shapiro, Stuart C.
N1 - Publisher Copyright:
© 2014 International Society of Information Fusion.
PY - 2014/10/3
Y1 - 2014/10/3
N2 - The area of hard+soft fusion is a relatively new topic within the information fusion community. One research effort which has confronted the subject of hard+soft fusion is the Multi-disciplinary University Research Initiative (MURI) titled Unified Research on Network-based Hard+Soft Information Fusion. Developed on this program is a fully integrated research prototype hard+soft fusion system in which raw hard and soft data are processed through hard sensor processing algorithms, natural language understanding processes, common referencing, alignment, association and situation assessment fusion processes. The MURI program is currently in its 5th (and last) year. During years 1 through 4, the MURI team dealt with the research issues in developing a baseline hard+soft fusion system, while identifying a number of design alternatives for each of the framework processing elements. For example, within natural language understanding different stemmers or ontologies could be utilized. The mathematical nature of hard or physical sensor processing and data association involved design choices about numerous parameters which affect the solution quality and solution quality/runtime tradeoff. While traditional experimental or training approaches may be used in assessing these processes in isolation, the nature and dependencies of hard+soft fusion require a systemic approach in which the integrated performance of framework components are understood. In this paper we describe the design of a test and evaluation framework for systemic error trail analysis and parametric optimization of hard+soft fusion framework sub-processes. We will discuss the performance metrics utilized including notions of system optimality, issues in defining the parametric space (design variants), cross-process error tracking methodologies and discuss some initial results. The presented system results are based on the Synthetic Counterinsurgency (SYNCOIN) dataset which is a dataset developed within the program and utilized for training and system optimization. Future work, including plans for the validation of experimental results will also be discussed.
AB - The area of hard+soft fusion is a relatively new topic within the information fusion community. One research effort which has confronted the subject of hard+soft fusion is the Multi-disciplinary University Research Initiative (MURI) titled Unified Research on Network-based Hard+Soft Information Fusion. Developed on this program is a fully integrated research prototype hard+soft fusion system in which raw hard and soft data are processed through hard sensor processing algorithms, natural language understanding processes, common referencing, alignment, association and situation assessment fusion processes. The MURI program is currently in its 5th (and last) year. During years 1 through 4, the MURI team dealt with the research issues in developing a baseline hard+soft fusion system, while identifying a number of design alternatives for each of the framework processing elements. For example, within natural language understanding different stemmers or ontologies could be utilized. The mathematical nature of hard or physical sensor processing and data association involved design choices about numerous parameters which affect the solution quality and solution quality/runtime tradeoff. While traditional experimental or training approaches may be used in assessing these processes in isolation, the nature and dependencies of hard+soft fusion require a systemic approach in which the integrated performance of framework components are understood. In this paper we describe the design of a test and evaluation framework for systemic error trail analysis and parametric optimization of hard+soft fusion framework sub-processes. We will discuss the performance metrics utilized including notions of system optimality, issues in defining the parametric space (design variants), cross-process error tracking methodologies and discuss some initial results. The presented system results are based on the Synthetic Counterinsurgency (SYNCOIN) dataset which is a dataset developed within the program and utilized for training and system optimization. Future work, including plans for the validation of experimental results will also be discussed.
KW - error audit trail
KW - evaluation metrics
KW - hard+soft information fusion
KW - system test and evaluation
KW - system under test
UR - http://www.scopus.com/inward/record.url?scp=84910681393&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84910681393&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:84910681393
T3 - FUSION 2014 - 17th International Conference on Information Fusion
BT - FUSION 2014 - 17th International Conference on Information Fusion
Y2 - 7 July 2014 through 10 July 2014
ER -