443
1 INTRODUCTION
One of the important carriers of the worldwide
economy is the transport of goods and persons
realized by vessels. The harmonization of the
developments of electronic aids to navigation and
dedicatedsystemsandservicesaboardandashorethe
International Maritime Organization (IMO) has
initiated the e‐Navigation strategy to int
egrate
existing and new navigational tools, in particular
electronictools,inanall‐embracingsystem.
The risk reduction of accidents between ships as
well as ships and obstacles is the social goal
associated to safe shipping from berth to berth. The
technological goal covers the development of new
toolsandmethodstosupporttheship‐sideandshore‐
side nautical staff during decision finding in
complicat
edandcomplexnavigationalsituations.
RelatedtotheSafetyofLifeatSeaConvention[1]
theprimarysourceforcollisionavoidanceandtraffic
situation awareness is the radar system with the
opportunitytodetectandtra
ckobjectswiththeRadar
PlottingAid(ARPA)functionalities[2].
With the implementation of the automatic
identification system (AIS) in 2004 an additional
importantstepwasdonetodeployasecondmeasure
for ship‐sideandshore‐sidevesseltracking[3].Like
almosteverytechnologies,neitherARPAnorAIScan
be declared as an “
altogether solution” and are
subjecttospecificrestrictionsandlimitations.Because
Radar Image Processing and AIS Target Fusion
F.Heymann,P.Banyś&C.Saez
DeutschesZentrumfürLuftundRaumfahrteV.,Neustrelitz,Germany
ABSTRACT:Collisionavoidanceisoneofthehigh‐levelsafetyobjectivesandrequiresacompleteandreliable
descriptionofmaritimetrafficsituation.Acombineduseofdataprovidedbyindependentdatasourcesisan
approachtoimprovetheaccuracyandintegrityoft
rafficsituationrelatedinformation.
Inthispaperwestudytheusageofradarimagesforautomaticidentificationsystem(AIS)andradarfusion.
Therefore we simulate synthetic radar images and evaluate the tracking performance of the particle filter
algorithmasthe most promising filter approach.Duringthe filterprocess the algorithmesti
matesthe target
position and velocity which we finally compare with the known position of the simulation. This approach
allowstheperformanceanalysisoftheparticlefilterforvesseltrackingonradarimages.Inasecondextended
simulation we add the respective AIS information of the target vessel and study the gained level of
improvementforthepart
iclefilterapproach.
The work of this paper is integrated in the research and development activities of DLR Institute of
CommunicationsandNavigationdealingwiththeintroductionofdataandsystemintegrityintothemaritime
trafficsystem.Oneoftheaimedobjectivesisthea
utomaticassessmentofthetrafficsituationaboardavessel
includingintegrityinformation.
http://www.transnav.eu
the International Journal
on Marine Navigation
and Safety of Sea Transportation
Volume 9
Number 3
September 2015
DOI:10.12716/1001.09.03.18
444
of the cooperative character of AIS data (disengage
able, dependent on the human initiated processes)
andthedependencyonotheronboarddevices(asfor
example the GPS receiver) there is still a margin for
errors in the data. Insofar the possibility cannot be
ruled out, that specific AIS data are
wrong or not
meaningfulduringimportantmaneuversofa vessel.
AnanalysisofacomprehensivetwomonthAISdata
set(JanuaryandFebruary2010)describingthevessel
trafficofthewholeBalticSea[4,5]cametoconclusion,
that specific parameters like Rate of Turn (ROT) as
wellasHeading(HDG)
deliversignificantlydefective
orimplausibleresults.Theradarontheothersideis
an electromagnetic sensor used for object detection
via reflected radio waves to determine the range,
altitude,direction,orspeedofobjects.Ifthemaritime
radar is installed with ARPA functionalities the
opportunityisgiventoderive tracksbased
onradar
targets.ARPA systems are able to calculate the
course and speed of tracked objects as well as the
closest point of approach (CPA) and time to closest
point of approach (TCPA) in relation to the own
vessel. The majority of ARPA systems integrate the
ARPAfeatureswiththe
radardisplay.
Previous studies of the maritime radar and the
ARPAsystem foundthat the ARPA drawbacks [6,7]
could be overcome with the use of the radar image
instead of distance and bearing calculated from
ARPA[8].Thispaperproposeandanalyzetheuseof
a sequential Monte Carlo algorithm also
known as
particlefilterasasolutionforradartargetextraction
andtrackingaswellasradarandAISfusion.
Themonitoringandassessmentofvesseltrafficis
an important element of safe, secure and efficient
shipping and the protection of environment. The
collision and grounding avoidance at sea requires
a
reliable and comprehensive picture of the maritime
traffic situation to enable an error‐free decision
making for the seafarers. A combined use of data
providedbyindependentdatasourcesisanapproach
to improve the accuracy and integrity of traffic
situation related information. This paper focuses on
theusageof
two‐dimensionalradarimagedataforan
improved target tracking in the frame of maritime
traffic monitoring. More precisely, the aim of this
paper is the analysis of a sequential Monte Carlo
methodforradartargetdetectionandtrackingaswell
as AIS and radar fusion. For this purpose the
paper
simulates radar images and AIS data to test the
proposedfilteralgorithm.
The paper is structured in the following way: At
firstinsection2thestrategyofstudyisdiscussed.In
the next part the scenarios and the generation of
synthetic images is described in section 3. Section 4
gives a very brief introduction into the used
sequential Monte Carlo method. In section 5 the
results are presented and section 6 discusses and
concludestheanalysisoftheresults.
2 THESTRATEGY
Aimofthis study is the performance analysis of the
sequential Monte Carlo method for target detection
andtrackinginmaritimeradarimageprocessing.The
strategyofthestudyisillustratedinFig.1andcovers
4steps.
Figure1.Systematicillustrationofthestrategy
Thefirststep of the study is the definition of the
test case scenarios. These scenarios are chosen such
thattheperformanceofthetrackingpositionaccuracy
and the time to first detection of the target can be
estimated. After the definition of the test cases the
sensor data has to
be simulated. This simulation
generateserrorfreeradarimagesaswellaserrorfree
AIS position data. After the data simulation the
sequentialMonteCarlomethodwasusedtoestimate
thepositionofthetarget.Thefirstpositionestimation
wasdoneinaradaronlymode.Inasecondstage
the
methodwasusedwithradarandAISdatainasensor
fusion process. The simulation environment has the
advantage that every parameter is precisely known
andthecomparisonofestimatedandsimulateddata
ispossible inorder to determine the performance of
the used method. The final step of the
study is the
analysisofresults.Duringtheanalysistheaccuracyof
the target position, extracted from the radar image,
and the time the algorithmneeds to extract the first
positionareestimated.
3 TESTCASEANDSENSORSIMULATION
Inthissectionwedescribethetestcasescenariosand
the
method used for radar image simulation. The
purpose of the scenarios is the performance
evaluationofthesequentialMonteCarlomethodfor
maritimeradarimageprocessing.
Thescenariosweredesignedtoestimatetheposition
accuracyoftheextractedtargetaswellasthetimethe
algorithm needs to calculate the first
position. The
simulationsweredoneforstaticanddynamicechoes
with and without AIS data. The first test case is a
static radar echo withoutAIS position data. For this
scenario different target echo sizes were simulated
and the position accuracy as well the time to first
positionfixwas
estimated.
Thesecondtestcaseisadynamicscenario.Inthis
scenariothetargetechomoveswiththevelocityvon
a straight line starting from position s
0 at time t=0
accordingthefollowingequation.
(1)
InthethirdscenariotheAISpositionasadditional
information is added to the static test case. The AIS
position is simulated at the center of the radar echo
without any additional position noise. In the last
scenario the AIS position is as well added to the
dynamicsimulation.
445
The following part describes the generation of
synthetic radar images, which are used as sensor
inputinthetestcasesimulations.
The generation of synthetic radar image is based
on data from a measurement campaign with the
vessel BALTIC TAUCHER II in the area of Rostock.
Duringthis campaignthe
actualscreen of the radar,
astheofficerofthewatchusesit,wasrecorded.The
radar images were extracted using a VGA to USB
grabberandthenstoredasaseriesofuncompressed
PNG‐images. The use of the original image source
solves the challenge of time synchronization and
allowsthesimulationarealisticsensorperformance.
OneoftheseimagesfromtheBALTICTAUCHER
IIradarscreen(SperryMarineVISIONMASTERFT)is
showninFig.2.
Figure2.TheoriginalRADARimage astakenfromSperry
MarineVISIONMASTERFTfromBALTICTAUCHERII.
Figure3.Thesimulatedradarimagewith5ellipticaltargets
ofdifferentsizeandorientation
The generation of synthetic images can be
described as follows. The process starts with the
removalof the part of the imagewhich contains the
radar echoes. After this process the synthetic image
consists only of the user interface and a filled black
circle. This was achieved by setting all pixel
values
withinaradiusof468pixelsaround the centerpixel
of (532,516) to zero. Note that this configuration is
specifictotheradardevicefromSperryMarine.After
the removal of the original data the synthetic radar
echoisaddedtotheimageasafilledyellowellipse.
The
use of ellipses for radar echoes is based on the
suggestionof[4]thatradarechoesarewelldescribed
withellipticalparameters.Theprocessisrepeatedfor
everyimagereceivedfromthedatastream.Whilethe
removal of the original radar echoes is always
identical the simulated target will be plotted
at the
positionderivedfromtheconfigureddynamicmodel.
Inprinciple,thisallowsthegenerationofanypossible
maritime scenario including near misses or even
collisions of traffic participants. In addition to the
simulationofradarechoesthesoftwareispossibleto
provideAISdataforallsimulatedvessels.Please
note
that the AIS dataset is derived from the simulated
radarechoeswithoutanyadditionalerroronposition
andvelocity.Additionallywewouldliketopointout
that the software is able to simulate targets with
elliptical shape, but the simulations were performed
with circular echoes of different sizes in order
to
reduce the complexity of the interpretation of the
results.
Thesimulated images are all based on data from
theSperryMarineVISIONMASTERFTRADAR,with
arange set to3 nautical miles. This configurationof
theradarresultsinthePixelsize11.87meters.
4 SEQUENTIALMONTECARLOSAMPLING:
PARTICLEFILTER
The paper studies the use of the sequential Monte
Carlo method for the position estimation of a target
vesselfromradarimagesandAISpositiondata.The
reasonforchoosingtheparticlefilterasthealgorithm
of choice is based on previous analysis of AIS and
radarsensors[8,9,10].
Figure4.Illustrationoftheparticlefilterprocesstakenfrom
[11]
Figure 4 illustrates the concept of the sequential
Monte Carlo method. A detailed description of the
algorithmisgivenin[11].Thealgorithmstartsattime
t‐1 with an un‐weighted distribution
i
1
t1
X,N
of
sampling particles (yellow circles in the first row of
446
Fig. 3). The next step is the calculation of the
importance weights
ii
t1 t1
X,w
of each particle
(bluecircles2ndrow).Theresultistheapproximation
of
t1 1:t1
p
(x | y )
. During the next step, the resampling
of the particles, only those particles are taken into
account which reproduces the observation best. The
result is an un‐weighted distribution of sampling
particles
i
1
t1
X,N
(third row yellow circles). The
finalpredictionstepusesthefiltermodeldescription
includingthemodelnoise,whichproducesvarietyin
the sampling distribution, for the approximation of
i
1
t
X,N
(4th row Fig. 3). The result is the
approximation of
t1:t1
p
(x | y )
(5th row Fig. 4),
which is the posterior distribution of the estimated
parameter. The process is repeated until the
simulationisfinished.
5 RESULTS
In this section the results of the simulations for the
static and dynamic scenarios with and without AIS
arepresented.
As already discussed two main performance
propertiesareofinterest.Thefirstistheconvergence
time,whichisthetimethemethodneedstoestimate
the position of the echo for the first time, and the
second is the tracking error E
T, which can be
interpreted as the position accuracy of the particle
filter.
C
ONVERGENCETIME
Initiallytheparticlesarespreadrandomlyoverthe
radar image. The filter has converged when the
majorityoftheparticlesareclosetothetarget.Atthis
pointtheparticlesmovewiththetargetandadoptits
shape.Inthisstudywedefineconvergencewhenthe
error, which is defined as
the Euclidean distance
between the simulated position of the target (Sx,Sy)
and the estimated position of the particle filter
(Px,Py), is smaller than 2 pixels. The distance is
calculatedwith
22
jjjjj
E Px Sx Py Sy, (2)
wherej isthe frame number and the position of the
simulated target and the position of particle filter
estimationisinradarcoordinates.
T
RACKINGERROR
Thetrackingerrorisameasureoftheaccuracyof
the algorithm. In this paper the tracking error is
defined, as the average over 400 images of the
Euclidean distance between the real position of the
target and the particle filter estimated position after
the algorithm has converged. The tracking error
ET
canbecalculatedwith:
C
N22
jj jj
jN
T
C
Px Sx Py Sy
E
NN
, (3)
whereNcistheconvergeframenumberasdefinedin
theconvergencesubsection,Nisthetotalnumberof
imagessynthesized,(Px,Py)ispositionoftheparticle
filter estimation and (Sx,Sy) is the position of the
simulatedtargetandjistheframenumber
Inthefollowingpartthe
resultsoftheperformed
simulationsarepresented.Theresults ofthe particle
filterprocesswithoutthefusionofAISdataisshown
first.ThereaftertheresultsoftheradarandAISfusion
arepresented.
S
IMULATIONWITHOUTAIS
This part presents the results for the simulations
without AIS. The first test scenario of a static radar
echoispresentedinFig.5.
Figure5. Tracking error as function of the number of
particles for the static simulation of radar echo with four
differentsizes
Thisfigureshowsthetrackingerrorinpixelsasa
functionofthenumberofparticlesforfourradarecho
sizes: 4 (black triangle); 8 (blue square); 12 (red
triangle)and14(greencircle)pixels.Thefigureshows
thatalowernumberofsamplingparticlesresultsina
highertracking error.
In additionit canbe seen that
largertargetsaremoredifficulttotrack,becausethey
showlargertrackingerrors.
Figure6.Trackingerrorasfunctionofthenumberoftarget
velocityforthedynamicsimulationofradarechoforthree
differentnumbersofsamplingparticles
The results of the dynamic test scenario without
AIS are shown in are shown in Fig. 6. The figure
shows the tracking error as a function of target
velocity for three different number of sampling
particles:10,000(black) 60,000(red) and
100,000(blue).
447
Thefigureshowsthatthetrackingperformanceof
the dynamic simulation can’t be improved by
increasing the number of sampling particles.
Additionally it can be seen that radar target echoes
withhighervelocityshowlargertrackingerrors.
S
IMULATIONWITHAIS
Figure7. Tracking error as function of the number of
particlesforthestaticsimulationwithandwithoutAISfor
fourdifferentechosizes
Fig.7showsthetrackingerrorinpixelsforparticle
filterconfigurationswithdifferentnumberofparticles
and radar echo sizes: 4 (black triangle); 8 (blue
square);12(redtriangle)and14(greencircle)pixels.
The figure compares the results of the static target
withAIS(coloredlines)withtheresults
presentedin
Fig.5withoutAIS(graylines).Asshowninthefigure
theadditionofAISinformationimprovesthetracking
performancebyafactorof2foralltargetechosizes.
But the tracking error of larger objects is still larger
thanforsmallerechoes.
Figure8.Trackingerrorasfunctionofthenumberoftarget
velocity for the dynamic simulation of radar echo for two
differentnumbersofsamplingparticles
Fig. 8 shows the results of the dynamic test
scenariowithandwithoutAIS.Thefigureshowsthat
thetracking error as function of target velocity does
not improve with the addition of AIS position data
and does not depend on the number of sampling
particles.
Figure9.Errordistanceasfunctionoftheframenumberfor
the dynamic simulation for two different veolcities with
additionalAISinformation.
Fig.9 shows the convergence time of the particle
filterforasimulationwithastatictargetoftwosizes
(4and8pixel)withandwithoutAISdata.Itisclearly
visible form this figure that the added AIS data
reduces the convergence time from 17s to 10s and
from
24sto10sforthetargetsofsizes4and8pixels,
respectively.
Figure10. Error distance as function of the frame number
forthedynamicsimulationfortwodifferentveolcities.
Fig.10showstheconvergencetimeoftheparticle
filterforasimulationwithadynamictargetwithAIS
data as solid lines and without AIS data as dashed
lines. This figure shows that the added AIS data
reduces the convergence time for the given scenario
by1sfortheslow
movingechoand2sforthefaster
one.
6 DISCUSSIONANDCONCLUSION
Inthis sectiontheresults ofthe previous sectionare
discussedandconcludingremarksaregiven.
This part discusses the results of the performed
simulations. The results of the static target scenario
show that the particle filter needs
for the first
detectionof the target radar echo at least10 frames,
which is equivalent to 5 antenna rotations. The
AdditionofAISdatadoesreducethisdetectiontime
byafactorof2.
The results of the dynamic target scenario show
larger tracking errors in comparison to the static
448
simulations. The explanation could be that the
tracking performance, which is as accurate as 0.5
pixels in the static simulations, is smaller than the
velocity of the vessel in the dynamic simulation.
Therefore it is possible that the larger tracking error
resultsfromthemovementofthetargetvesselinstead
of the position estimation. This would explain the
resultofatrackingerrorsimilartothevesselspeedof
1pixel/second.Thisisconsistentwiththefactthatthe
dynamic simulations with smaller velocities show
smaller tracking errors. The results of very slow
moving vessels show the same particle filter
performanceasthenon‐movingstatictargets.
Additionally we like to point out that the filter
tuningisanimportantpartoftheoverallperformance
ofthefilter.
Inthefollowingpartweconcludetheresultsofthe
study.In this paper we simulated scenarios of static
and dynamic radar targets.
The simulations were
usedtoestimatetheradartrackingperformanceofa
sequentialMonteCarlofiltertofollowmaritimeradar
echoes. The performed simulations cover situations
withandwithoutadditionalAISsensorpositiondata
inthefusionprocess.
Theresultscanbesummarizedasfollows.
The time to target
vessel echo detection is
smallerthan1minute
The resulting tracking accuracy of sequential
Monte Carlo method is smaller than 1 pixel (~10
meters)
The model assumption used in the particle filter
hasastrongimpactontheresultingperformance
The addition ofAIS data increase the
performance
thefusionprocesssignificantly
Theinthis paper performedsimulations strongly
suggest that the sequential Monte Carlo method is
suitableforAISandradarimagefusion.
The next planned step is the improvement of the
sequential Monte Carlo filter simulation to more
realistic scenarios. This enhancement of the
simulation will
improve the performance evaluation
of the fusion of radar image and AIS position. This
stepisnecessarytogainintegrityinformationfroman
AISandradarfusionprocesswiththefinalaimofthe
introduction of data and system integrity into the
maritime traffic situation aboard a vessel as well as
ashore.
REFERENCES
[1]SOLAS. 1974. ‘Safety of Life at Sea Conventions ’,
SOLAS
[2]IMO RESOLUTION A.823(19). 1995. “PERFORMANCE
STANDARDS FOR AUTOMATIC RADAR PLOTTING
AIDS(ARPAs).”
[3]Rec. ITU‐R M.1371‐3 ‘Technical Characteristics for an
Automatic Identification System Using Time Division
MultipleAccessintheVHFMaritimeMobileBand.’
[4]P.Bany
ś.,T.Noack,S.Gewies(2012),‘ASSESSMENTOF
AISVESSELPOSITIONREPORTUNDERTHEASPECT
OFDATARELIABILITY’,ProceedingsoftheEuropean
NavigationConference2012,Gdynia
[5]F. Heymann, T. Noack, P. Banyś (2013), ‘Plausibility
analysis of navigation related AIS parameter based on
time series’, Proceedings of the European Navigation
Conference2013,Vienna
[6]P. Banyś, E. Engler, F. Heymann, and T. Noack (2013)
‘Timestamp Discrepancies in Multisensor NMEA
EnvironmentduringSurveyVoyage’,ScientificJournals
MaritimeUniversityofSzczecin,36(1),pp.22‐26.
[7]A.Lenart(1989),“ARPAAccuracyTesting.”, Journalof
Navigation,42,pp.117‐123
[8]F.
Heymann,P.Banyś,T.Noack(2014),‘Apilotstudyof
the advantage of radar image data over ARPA‐based
position and bearing’, Proceedings of the European
NavigationConference2014,Rotterdam
[9]F. Heymann, P. Banyś., T. Noack, E. Engler (2013), ‘Is
ARPA Suitable for Automatic Assessment of AIS
Targets?’,AdvancesinMarineNavigation,pp.223‐232
[10]L.Tong,F.Heymann,andT.Noack(2013)‘Radartarget
detectionbasedonmethodsofimagepatternmatching’,
Scientific Journals Maritime University of Szczecin, 36
(108),pp.162‐167.
[11]A. Doucet, N. Freitas, N. Gordon (2001), “An
Introduction to Sequential Monte
Carlo Methods”,
SequentialMonteCarloMethodsinPractice,NewYork:
Springer‐Verlag,January2001
