Systemic test and evaluation of a hard+soft information fusion framework: Challenges and current approaches

Geoff A. Gross, Ketan Date, Daniel R. Schlegel, Jason J. Corso, James Llinas, Rakesh Nagi, Stuart C. Shapiro

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

14 Scopus citations

Abstract

The area of hard+soft fusion is a relatively new topic within the information fusion community. One research effort which has confronted the subject of hard+soft fusion is the Multi-disciplinary University Research Initiative (MURI) titled Unified Research on Network-based Hard+Soft Information Fusion. Developed on this program is a fully integrated research prototype hard+soft fusion system in which raw hard and soft data are processed through hard sensor processing algorithms, natural language understanding processes, common referencing, alignment, association and situation assessment fusion processes. The MURI program is currently in its 5th (and last) year. During years 1 through 4, the MURI team dealt with the research issues in developing a baseline hard+soft fusion system, while identifying a number of design alternatives for each of the framework processing elements. For example, within natural language understanding different stemmers or ontologies could be utilized. The mathematical nature of hard or physical sensor processing and data association involved design choices about numerous parameters which affect the solution quality and solution quality/runtime tradeoff. While traditional experimental or training approaches may be used in assessing these processes in isolation, the nature and dependencies of hard+soft fusion require a systemic approach in which the integrated performance of framework components are understood. In this paper we describe the design of a test and evaluation framework for systemic error trail analysis and parametric optimization of hard+soft fusion framework sub-processes. We will discuss the performance metrics utilized including notions of system optimality, issues in defining the parametric space (design variants), cross-process error tracking methodologies and discuss some initial results. The presented system results are based on the Synthetic Counterinsurgency (SYNCOIN) dataset which is a dataset developed within the program and utilized for training and system optimization. Future work, including plans for the validation of experimental results will also be discussed.

Original languageEnglish
Title of host publicationFUSION 2014 - 17th International Conference on Information Fusion
ISBN (Electronic)9788490123553
StatePublished - 3 Oct 2014
Event17th International Conference on Information Fusion, FUSION 2014 - Salamanca, Spain
Duration: 7 Jul 201410 Jul 2014

Publication series

NameFUSION 2014 - 17th International Conference on Information Fusion

Conference

Conference17th International Conference on Information Fusion, FUSION 2014
Country/TerritorySpain
CitySalamanca
Period7/07/1410/07/14

Keywords

  • error audit trail
  • evaluation metrics
  • hard+soft information fusion
  • system test and evaluation
  • system under test

Fingerprint

Dive into the research topics of 'Systemic test and evaluation of a hard+soft information fusion framework: Challenges and current approaches'. Together they form a unique fingerprint.

Cite this