Improving the Fusion Process Using Semantic Level Whole-Brain Analysis

David L. Hall, M. D. McNeese, Eileen Rotthoff, Tim Shaw, James Z. Wang
The Pennsylvania State University

Extensive research in the fusion community has focused on developing techniques for level 1 fusion. The bulk of the literature on multisensor data fusion focuses on the automation of target tracking and automatic target recognition. While such research is needed, current problems involve complexities such as identifying and tracking individual people and groups of people, monitoring global activities and recognition of events that may be a precursor to terrorist activities. The requisite data for this analysis involves sensor data (including signals, images, and vector quantities), textual information (from web sites and human reports), and utilization of models. This analysis process is very human intensive, requiring teams of analysts to search for data, interpret the results, develop alternative hypotheses, and assessment of the consequences of such hypotheses. This paper describes the development of a set of tools aimed at level-5 fusion to support whole-brain analysis. The toolkit encourages analysts to use and integrate both their visual intelligence and their language processing ability. We have developed a set of tools to support multi-INT analysts including tools for; (1) automated semantic labeling of images, (2) collaboration tools, (3) cognitive aids, (4) advanced visualization aids, and (5) context-based interpretation techniques. These tools are being implemented in an open architecture involving a commercial off the shelf geographical information system (GIS). In addition to the tool kit, a living laboratory environment has been developed to allow the quantitative evaluation of these aids in a repeatable, experimental environment.

Full Paper in Color
(PDF, 2.2MB)   

On-line info   

Citation: David L. Hall, M.D. McNeese, Eileen Rotthoff, Tim Shaw, and James Z. Wang, ``Improving the Fusion Process Using Semantic Level Whole-Brain Analysis,'' Proc. MSS National Symposium on Sensors and Data Fusion, 7 pages, Monterey, CA, May 2005.

Copyright 2005. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works, must be obtained from the publisher.

Last Modified: Mon Jul 18 15:41:24 EDT 2005
2005, James Z. Wang