International Journal
on Marine Navigation
and Safety of Sea Transportation
Volume 2
Number 1
March 2008
85
Multisensor Data Fusion in the Decision
Process on the Bridge of the Vessel
T. Neumann
Gdynia Maritime University, Gdynia, Poland
ABSTRACT: More and more electronic devices appears on the bridge of the vessel. All of them are supposed
to help navigator in his work. Some of them are useful for exchanging data among vessels. Nowadays
navigator can observe surroundings of the vessel on screens of some different systems of exchanging data. It
is obvious that there are some advantages and some disadvantages of each of these systems. Proposal of the
author is connecting data obtained from mentioned systems by means of data fusion technique. Joining few
systems in one will be helpful at making decision on the bridge of the vessel. This paper is an introduction to
consideration how to use the data fusion in the maritime navigation.
1 INTRODUCTION
1.1 Systems of the exchange of data
The scientific and technological progress is bringing
some new solutions. There are more and more
electronic devices on the vessel’s bridge. That
cause1 navigator has the access to various systems of
the exchange of data. Some of them can receive data,
other combines send-receive operation.
The navigator’s assessment of collision risk
depends on his knowledge about own ship’s motion
and other ships’ motion. The available means for
assessing the other ships’ motion are for example:
visual sighting, radar, ARPA, AIS and the voice
communication with other ships. Each of
enumerated systems possesses particular reliable
features.
Voice communication, radar and visual sighting
give real time information. Each of them is a
separate system on the bridge of the vessel. The most
difficult for the navigator can be predicting the
situation in advance if the safety margins are small,
as in congested waters. The same applies for
Automatic Identification Systems (AIS) if only the
text display is provided. It is appeared, that the AIS
will be able to replace many of enumerated means of
communication.
Fig. 1. Some systems of exchanging data on the bridge of the
vessel
86
Very important question is possibility to switch
off AIS receiver. Acts of piracy represent a serious
threat to the lives of seafarers and the safety of
navigation. In such situation switched AIS is making
vessel to be sitting target. Of course sometimes AIS
receiver should be switched off.
It is appeared, that the AIS and ARPA can
collaborate with themselves. AIS, if works in the
graphical mode, have the advantage that its results
easy to interpret and it is easy to predict the other
ships’ motion based on the information available at
the moment.
The AIS is known as a system providing other
ships’ course and speed in real time, in opposed to
the ARPA system which calculates the course and
speed from historic radar data. For this reason it may
be suspected that information obtained from the AIS
in many cases will be less reliable than information
from the ARPA.
Of course, in some situation AIS can also provide
incorrect data. In this system the course and speed
over ground may be provided from a GPS with very
slow filters. This may cause the AIS course and
speed information to be more delayed and less
accurate than the ARPA calculated information.
It is possible to connect all systems of the
exchange of data which are found on the bridge of
the vessel into one system. Each of enumerated
systems will be still working individually.
This paper presents theoretical rules about joining
similar data from different sources.
2 A DATA FUSION PROCESS MODEL
Data fusion means a very wide domain and it is
rather difficult to provide a precise definition.
Several definitions of data fusion have been
proposed. Pohl and Van Genderen (Wald, 1999)
defined “ image fusion is the combination of two or
more different images to form a new image by using
a certain algorithm” which is restricted to image.
Hall and Llinas (Wald, 1999) defined “data fusion
techniques combine data from multiple sensors, and
related information from associated databases, to
achieve improved accuracy and more specific
inferences that could be achieved by the use of
single sensor alone”. This definition focused on
information quality and fusion methods. According
to these definitions, it could imply that purposes of
data fusion should be the information obtained that
hopefully should at least improve image visualiza-
tion and interpretation.
The basic definition of data fusion is as follow:
“combining information to estimate or predict the
state of some aspect of the world”.
General steps in data fusion process are shown at
fig. 2. In the process it is possible to appoint such
steps as data receiving, pre-processing, fusion and
visualisation.
Fig. 2. Data Fusion Process
There are several fusion approaches. Generally
fusion can be divided into three main categories
based on the stage at which the fusion is performed
namely:
− pixel based,
− feature based,
− decision based.
In pixel based fusion, the data are merged on a
pixel-by-pixel basis.
Feature based approach always merge the
different data sources at the intermediate level. Each
image from different sources is segmented and the
segmented images are fused together.
Decision based fusion, the outputs of each of the
single source interpretation are combined to create a
new interpretation.
In the scheme, shown in fig. 3, the data fusion
process is conceptualized by sensor inputs, human-
computer interaction, database management, source
87
pre-processing, and four key sub-processes.
Sometimes data fusion domain includes two
additional sub-processes (Level 0 and Level 5).
3 PHASES OF DATA FUSION PROCESS
The best known model of data fusion functions is the
JDL (Joint Directors of Laboratories) model. Its
differentiation of functions into fusion levels
provides a useful distinction among data fusion
processes that relate to the refinement of “objects,”
“situations,” “threats,” and “processes.”
3.1 Level 0 - Sub-Object Data Association and
Estimation
This level is not very often included in data fusion
domain. There is a data processing on the signal
level in this phase.
3.2 Level 1 - Object Refinement
The main task of this level is combining data from
multiple sensors and other sources to determine
position, kinematics, and other attributes.
The first general method of combining multi-
sensor data, known as data association, correlates
one set of sensor observations with another set of
observations. As a result of this process, data
association is able to produce a set of “tracks” for a
target object. A track is an estimate of a target’s
kinematics, including such factors as its position,
velocity, and rate of acceleration (Hughes, 1989).
Thus, data association represents the initial step
necessary for localizing a target; this can later be
increased with the identification of other characteris-
tics associated with the target.
In tracking targets with less-than-unity probability
of detection in the presence of false alarms, data
association is crucial. A number of algorithms have
been developed to solve this problem. Two simple
solutions are the Strongest Neighbour Filter (SNF)
and the Nearest Neighbour Filter (NNF). In the SNF,
the signal with the highest intensity among the
validated measurements is used for track update and
the others are discarded. In the NNF, the
measurement closest to the predicted measurement is
used.
Data association becomes more difficult with
multiple targets where the tracks compete for
measurements. Here, in addition to a track validating
multiple measurements as in the single target case, a
measurement itself can be validated by multiple
tracks. Many algorithms exist to handle this
contention. The Joint Probabilistic Data Association
(JPDA) algorithm is used to track multiple targets by
evaluating the measurement-to-track association
probabilities and combining them to find the state
estimate. The Multiple-Hypothesis Tracking (MHT)
is a more powerful (but much more complex)
algorithm that handles the multi-target tracking
problem by evaluating the likelihood that there is a
target given a sequence of measurements (Hall,
1989).
O
U
R
C
E
HUMAN
COMPUTER
INTER-
ACTION
SOURCE
PRE-
PROCESSING
LEVEL 0
SIGNAL
REFINEMENT
LEVEL 1
OBJECT
REFINEMENT
LEVEL 2
SITUATION
REFINEMENT
LEVEL 3
CRITICAL
REFINEMENT
LEVEL 4
PROCESS
REFINEMENT
LEVEL 5
COGNITIVE
REFINEMENT
DATABASE MANAGEMENT SYSTEM
SUPPORT
DATABASE
FUSION
DATABASE
Fig. 3. The Joint Directors of Laboratories data fusion model (Adapted from Hall & McMullen, 2004)
88
3.3 Level 2 - Situation Refinement
Level two data fusion represents an advance beyond
the creation of raw sensor data, as occurs at the first
level, and supports the synthesis of more meaningful
information for guiding human decision-making.
Bayesian decision theory is one of the most common
techniques employed in level two data fusion. It is
used to generate a probabilistic model of uncertain
system states by consolidating and interpreting
overlapping data provided by several sensors. It also
determines conditional probabilities from a priori
evidence.
On this level is used one of two most popular
techniques which are:
− Bayesian Decision Theory
− Dempster-Shafer Evidential Reasoning
3.3.1 Bayesian Networks
Bayesian networks are useful for both inferential
exploration of previously undetermined relationships
among variables as well as descriptions of these
relationships upon discovery.
3.3.2 Dempster-Shafer evidential reasoning (DSER)
The Dempster-Shafer method has several other
advantages over Bayesian decision theory. Most
importantly, hypotheses do not have to be mutually
exclusive, and the probabilities involved can be
either empirical or subjective. Because DSER sensor
data can be reported at varying levels of abstraction,
a priori knowledge can be presented in varying
formats. It is also possible to use any relevant data
that may exist, as long as their distribution is
parametric.(Hughes, 1989).
3.4 Level 3 - Critical Refinement
Level 3 processing projects the current situation into
the future to draw inferences about threats and
opportunities for operations (Hall, 1989)
On this level is used one of three most popular
techniques which are:
− Expert Systems,
− Blackboard Architecture,
− Fuzzy Logic.
3.4.1 Expert Systems
An expert system is regarded as the
personification within a computer of a knowledge-
based component from an expert skill in such a form
that the system can offer intelligent advice or take an
intelligent decision about processing function.
3.4.2 Blackboard Architecture
A blackboard-system application consists of three
major components:
− The software specialist modules, which are called
knowledge sources. Like the human experts at a
blackboard, each knowledge source provides
specific expertise needed by the application.
− The blackboard, a shared repository of problems,
partial solutions, suggestions, and contributed
information.
− The control shell, which controls the flow of
problem-solving activity in the system.
3.4.3 Fuzzy Logic
Fuzzy Logic is a mathematical technique for
dealing with imprecise data and problems that have
many solutions rather than one.
Fuzzy logic is derived from fuzzy set theory
dealing with reasoning that is approximate rather
than precisely deduced from classical predicate
logic.
Level 2 and Level 3 fusion are very challenging.
They involve the attempt to emulate human
reasoning.
3.5 Level 4 – Process Refinement
Level 4 was defined as a meta-process. The process
monitors the data fusion process and tries to
optimize the process by controlling the sensor
resources in order to achieve improved fused results.
Basically the purpose of sensor management is to
optimize fusion performance by managing the sensor
resources. It can therefore be considered as a
decision making task, taking viewpoint from
decision theory, determining the most appropriate
sensor action to be taken in order to achieve
maximum utility. (Xiong and Svensson, 2003).
3.6 Level 5 – Cognitive Refinement
According to Hall & McMullen (2004) human-
computer interaction (HCI) research in the fusion
domain has mainly considered interaction between
the user and a geographical information display
(based on a geographical information system)
through menus and dialogs. However, the current
research interest in this area is growing, and
techniques such as gesture recognition and natural
language interaction are currently of interest.
89
4 REMARKS
In this paper there were presented some different
systems of the exchanging data among vessels. It
contains also descriptions of situations when similar
data coming from different systems can cause
making wrong decisions. One method which can be
used to analyze data in these situations is data fusion
method presented above. It is appeared that using
technique of data fusion can enable navigator to
solve complex problems concerning choosing the
most available route of vessel.
REFERENCES
Andler, S. F. Information Fusion from Databases, Sensors and
Simulations, Annual Report 2005, June 2006.
Hall, D. & Llinas, J. Handbook of multisensor data fusion.
CRC Press. Hughes, T.J. “Sensor Fusion in a Military
Avionics Environment.” Measurement and Control. Sept.
1989.
Hall, D. & McMullen, S.A.H. (2004) Mathematical techniques
in multisensor data fusion. Artech House.
Hughes, T.J. “Sensor Fusion in a Military Avionics
Environment.” Measurement and Control. Sept. 1989:
Ramsvik, H. AIS as a tool for Safety of Navigation and
Security - Improvement or not?
Svensson, P. Technical survey and forecast for information
fusion. In: RTO IST. Symposium on Military Data and
Information Fusion. 20-22 October, 2003.
Wald L., 1999, Some Terms of Reference in Data Fusion, IEEE
Transactions on Geoscience and Remote Sensing Vol.37
No.3 May 1999.
