1 INTRODUCTION

Autonomous ships are receiving significant attention

from the academic community and industry in recent

years. The new upsurge will have a profound impact

on maritime industry to great extent, in which it will

affect shipping companies, maritime operations,

shipbuilders and their operational mode. In 2012 and

2015, the funding of the "Maritime Unmanned

Navigation through Intelligence in Networks

(MUNIN)" project by the European Union [1] and

Rolls-Royce led "Advanced Autonomous Waterborne

Applications Initiative (AAWA)" [2] to outline the

concept of autonomous ships and the vision of

turning remote and autonomous shipping into a

reality. At the same time, International Maritime

Organization (IMO) also take some steps to

investigate safety, security and legal issues for

autonomous ships in IMO instruments [3-6, 29].

Regardless of the developing stage of autonomous

ships, the key issue needs to be emphasized is that

the ship be capable of an equivalent level of safety to

the conventional ships.

The ability of a ship to monitor its own health,

establish and communicate what is around it and

make decisions based on that information is vital to

autonomous operations. At the current stage, how to

better achieve autonomous navigation has become a

top priority. Fully autonomous navigation for the

Quantitative Processing of Situation Awareness for

Autonomous Ships Navigation

X.Y. Zhou

Dalian Maritime University, Dalian, China

National University of Singapore, Singapore

Z.J. Liu, Z.L. WU & F.W. Wang

Dalian Maritime University, Dalian, China

ABSTRACT: The first ever attempt at fully autonomous dock-to-dock operation has been tested and

demonstrated successfully at the end of 2018. The revolutionary shift is feared to have a negative impact on the

safety of navigation and the getting of real-time situation awareness. Especially, the centralized context

onboard could be changed to a distributed context. In navigation safety domain, monitoring, control,

assessment of dangerous situations, support of operators of decision-making support system should be

implemented in real time. In the context of autonomous ships, decision-making processes will play an

important role under such ocean autonomy, therefore the same technologies should consist of adequate system

intelligence. At the same time, situation awareness is the key element of the decision-making processes.

Although there is substantial research on situation awareness measurement techniques, they are not suitable to

directly execute quantitative processing for the situation awareness of autonomous ships navigation. Hence, a

novel quantitative model of situation awareness is firstly proposed based on the system safety control structure

of remotely controlled vessel. The data source is greatly limited, but the main result still indicates that the

probability of operator lose adequate situation awareness of the autonomous ship is significantly higher than

the conventional ship. Finally, the paper provides a probabilistic theory and model for high-level abstractions

of situation awareness to guide future evaluation of the navigation safety of autonomous ships.

http://www.transnav.eu

the International Journal

on Marine Navigation

and Safety of Sea Transportation

Volume 13

Number 1

March 2019

DOI: 10.12716/1001.13.01.01

duration of the whole voyage is extremely difficult to

realize in recent years, which based on existing level

of technological development. Remote-controlled

mode will be a feasible solution. However, removal

of vessels from direct control and the visual field of

operator inevitably reduces their ability to directly

attain adequate situation awareness (SA) of the vessel

and its surroundings.

SA is a prerequisite to rational decision-making in

many contexts, from individual operator to team

cooperation, and reflects the ability of a human to

perceive elements in their environment, comprehend

their meaning and project their state in the near

future [7]. It plays a crucial role in effective risk

reduction for operators. Currently, because of the

complexity of real ship maneuvering system, there

are few research and method are applied to enhance

SA successfully in the maritime industry. Previous

research shows that the training procedure using

simulator is an positive way to improve SA,

operational effectiveness and safety [25]. In the

relatively advanced training framework, different

information can be collected from the simulator scene

and from the real world to provide data support

(Figure 1), such as audio, video, bio-metric data from

eye-trackers. Furthermore, the application of

Wearable Immersive Augmented Reality (WIAR)

technology [26] also offer a new solution of next

generation navigation system, enhanced and remote

monitoring, and improve operator performance.

Figure 1. The gaze plot when operator wearing eye-tracker

However, the emergence of the concept of

autonomous ships in remote-control mode, the

complexity in autonomous navigation systems

reaching a new high-level, and some unnecessary

complexity adds to captains' or operator's confusions

as it changes the context of the interaction. For

example, captain's responsibility from the bridge to

shore-based control centre (SCC), and the decision

support system is integrated to maintain safety of

navigation. For this reason, the centralized context

onboard would shift to a distributed context [27]. In

the substantive research of SA, the physics and

cognition of operators are considered to be fixed in a

centralized system. And few are paying attention to

how SA might be influenced in distributed working

domains. In such system, the operator needs to

maintain an adequate situation understanding to

ensure the safety of ships, which is critical.

At present, the research of remotely-controlled

ships navigation possesses very little in the way of

shore-based situation awareness and focus on

qualitative analysis. And the design of the system is

still being developed and the final structure remains

uncertain, therefore it is difficult to explore all the

possible scenarios that may arise from the

combination of components' behavior [8]. Any

inaccuracy situational representation will propagate

into decision-making process. When adding

autonomy in order to increase situation aware-ness, it

is important to take into account the possibility that

operators located on shore may be unfamiliar with

the technology or uncertain over its capability [9],

and over-reliance may skew the decision-making

process.

In order to maintain an equivalent safety level,

better serve the construction of the remote control

system and the training of shore-based operators, it is

necessary to quantify the SA in autonomous ships

navigation.

In this paper, we propose a novel model for

quantifying the SA of autonomous ships navigation

focuses on "remote control" mode. The next section

lays down the overview of autonomous ships and

determine the object in this paper. Section III

introduces the theoretical background of SA and

discuss the current SA measurement techniques.

Section IV proposes a new quantitative method for

modeling the SA of autonomous ships navigation

considering the probability with each known

awareness element. Section V discusses the result of

model. The paper ends with conclusions and

potential future works in section VI.

2 THE OVERVIEW OF AUTONOMOUS SHIPS

The idea of autonomous ships is partly derived from

the Unmanned Surface Vehicle (USV), and the

concept of autonomous ship was first described by

Schönknecht in 1983 [10]. Subsequently, Japan

explored this concept in more depth to minimize

crew costs and built several automated ships. Recent

technological advancements in big data and

communication infrastructure, intelligent ships have

entered people's field of vision as the highest level of

automation. Compared with conventional ship,

autonomous ship will be a highly integrated ship of

various systems, which is the advanced stage of

intelligent ship development.

According to the design concept of autonomous

ships, nearly all subsystems of autonomous ships will

be controlled by remote or autonomous mode,

including collision avoidance decision-making and

ship state monitoring. The vessel may be manned

with a reduced crew or unmanned with or without

supervision and have the capabilities to make

decisions and perform actions with or without

human in the loop. To a varying degree, it can

operate independent of human interaction. In

general, the control mode can be divided into four

categories (Figure 2). The remotely-controlled

merchant vessel will play an important role for

maritime transportation system, and become a reality

in recent years, even though there are many

challenges. In this paper, we focus on the "remote

control" mode supported by SCC.

Figure 2. The development stages of autonomous ships

3 THEORETICAL BACKGROUND AND

MEASUREMENT OF SITUATION AWARENESS

In the early 1980s, the situation theory was developed

by Jon Barwise [11-13] and then was successfully

extended and application [14, 15] in many domains,

especially in the military. There are various

definitions and understandings of SA. And most

popularly cited one and firstly introduced by Endsley

[16]. The summarized definitions that SA is

perception of element of the environment within a

volume of time and space (level 1 SA), the

comprehension of the current situation (level 2 SA)

and the projection of the status in the near future

(level 3 SA). Figure 3 shows Endsley's three level SA

models. The increased use of teams in complex

environments has shifted the focus from individual

operator SA onto the shared SA of teams of

operators, which to deal with multiple information

resources. Team or shared SA reflects the coordinated

awareness that the team possesses as a whole unit

[16, 17]. In this paper, we mentioned "team SA" as a

generalization of individual SA, and be defined as the

sum of the technical and non-technical skill of each

member of the team.

Across the domains, an initial literature review

was conducted in order to create an exhaustive

database of existing SA measurement techniques, and

determine whether any of these approaches could

potentially be used in the assessment of SA in

autonomous ships navigation. More than 30 different

measurement techniques of SA have been identified

and can be generally categorized as direct measures

and indirect measures. Due to the limited space of

this article, the following categories of SA

measurement techniques were enumerated.

1 SA requirement analysis; including unstructured

interviews and structured questionnaires [18].

2 Freeze probe techniques; such as Situation

Awareness Global Assessment Technique

(SAGAT) [16].

3 Real-time probe techniques; such as Situation

Awareness Assessment Method (SPAM) [19].

4 Self-rating techniques; such as Situation

Awareness Rating Technique (SART) [20],

Situation Awareness Rating Scales Technique

(SARS) [21] and Crew Awareness Rating Scale

(CARS) [22].

5 Observer-rating techniques; such as Situation

Aware-ness Behavioral Rating Scale (SABARS)

[23].

6 Performance measures.

7 Process indices; typical measurement techniques

such as eye tracker, verbal protocol analysis, etc.

Among them, SAGAT and SART approaches are

by far the most commonly applied during individual

and team SA assessment. SAGAT offer a direct

measurement way of operator SA, which removes the

numerous problems associated with collecting post-

trial and subjective SA data. However, a real scenario

with multiple information sources was frozen and

managing SA queries to multiple agents in

distributed SA model to be almost impossible. And

the method carries a high level of intrusion upon

primary task performance caused by the task freezes

[28]. For SART, it is non-intrusive to task

performance and can be obtained from different team

members, but participants may not be able to

accurately rate the level of SA when they have

inadequate SA.

Figure 3. Endsley's decision-making, information

processing and SA model

The existing SA measurement techniques is

difficult to meet the requirements to assess SA across

multiple locations at the same time, both individual

and team SA for the same task and also assess SA in

real time. Therefore, to properly assess how SA is

influenced in a distributed ship-shore context during

the tasks of monitoring and controlling autonomous

ships, and quantify the observability and

understandability in the human-automation

interaction process would be affected intrinsically for

the task of remote monitoring and controlling vessels,

a novel and more quantitative approach need to be

proposed which considers not only the attainment of

elements of awareness but also their quality and

reliability.

4 THE PROPOSED QUANTITATIVE MODEL

The fact that the remotely controlled merchant

vessels will affect virtually all aspects of her

operation, including navigation. Operators and full

bridge team will be located in SCC to oversee

decision making, supervision and trouble-shooting.

Here, an operator located onshore will have an

overall command over a handful of vessels traversing

different seas. However, the full bridge team can

provide assistance to better deal with the problem, as

soon as a situation develops in a significant difficulty

and emergency. Because of the inherent uncertainty

of quantifying these properties, we seek to calculate a

probability of loss of SA, which can be presented as

follow:

()1 ()PSA PSA , (1)

where

()PSA is the probability of a system not

possessing adequate awareness.

The adequate SA is based on the accurate

knowledge of systems and a genuine experience of

the vessels' current state via multiple source

information. In this paper, we do not attempt to

define these performance criteria or even assess a

particular element. Instead, a Bayesian inference

based quantitative model is presented to quantify the

awareness of autonomous ships navigation in

abstract terms. In order to clarify the composition of

remotely-controlled merchant vessel, we adopt the

safety control structure was first developed by

Wróbel [24] (Figure 4). The safety control structure is

focus on the “remote control” mode of autonomous

ships and built under Systems Theoretic Process

Analysis (STPA) framework, which is a novel hazard

identification method. Therefore, it is satisfying the

requirement of quantitative SA model.

Figure 4. The system's safety control structure of remotely-

controlled vessel [24]

The structure was divided into five parts, namely

organizational environment, shore facilities (SCC),

communication, vessel and environment. Data was

supported by various sensors is used to provide SA

for operator on shore to make system-level decisions.

These sensors can be either environmental or

internal. While the former will obtain the

navigational situation around the vessel, and the

latter will monitor the interior of vessel. Besides, the

company managers will evaluate and audit the

performance of operator. Meanwhile, the company

managers will consistently imp-rove the system with

the feedback from the organizational environment,

the experience sharing by the operators in the form of

reports and meetings as well as their own

interpretations. As such, the operational procedures

are updated, prescriptive advices are inputted and

the operators' ability of SA is enhanced.



defined as the impact of the company mangers on the

operator. When





, the impact is positive,

otherwise is negative. The relationship with the

probability of the operator of autonomous ships

maintain adequate SA during navigation can be

expressed as

() ( )(1 )

PSA PSA



, (2)

where

()PSA is the probability of a system possess

adequate SA, and

()

PSA is the probability that the

operator maintain adequate SA during navigation. It

should be noted that an operator may simultaneously

full-control more than one vessel in the safety control

structure of remotely-controlled merchant vessel.

Abstractly, we assumed that an operator can

overall command over

vessels in the fleet.

However, the probability that the operator

adequately percept elements in the current situation,

comprehend the current situation and project future

status is different from different vessels. In this

paper, an operator's ability of SA can be seen as a

team SA consisted with n single vessels. The

current situation and communication link of different

vessels has a huge gap. It is unscientific and

unreasonable to simply use arithmetic mean,

geometric mean and harmonic mean to calculate

()

PSA . Such these simple averaging methods would

partly ignore the serious influence of low SA ability

from a vessel. Therefore, we need to consider the

interaction of each remotely-controlled ship on the

operator. The mental models of the operator cannot

be updated and reset when the operator continuously

control different single vessel or manage the fleet at

the same time. And the negative effects of multi-

tasking interactions cannot be easily ignored. Based

on the above analysis, an effective method for

calculating team SA was proposed, which

considering the interaction of multi-tasking.

()

PSA

can be represented by the

()

PSA , which is given by



() () ()( )

oii

PSA PSA PSA PSA







  



, (3)

where

()

PSA is the probability that an operator

possess adequate SA with a single vessel. The

example of shared (team) SA is demonstrated in

Figure 5.

Figure 5. An example of shared (team) SA

The figure shows that the sum of the cross section

of each vessel's SA weighted by numbers of vessels

which can share confidence.

()PSA ,

()PSA and

()PSA are assumed to be 0.1, 0.1, 0.6

and 0.8. The shared portion for all vessels is 0.1, and

the shared portion for two vessels (SA

3, SA4) is 0.5.

The portion of SA that one vessel owns alone does

not count.

Under the safety control structure of remotely-

controlled vessel,

()

PSA should be quantitative

calculation. Therefore, a novel theoretical setting

based on the mathematical frame-work of

Hierarchical Bayesian Inference was proposed. In this

model,

()

PSA will be described by its complement.

A probabilistic description of effects of different

elements on one another is given by

()1 ()

PSA PSA

, (4)

()()()()()

iivvivv

P SA P SA SA P SA P SA SA P SA, (5)

where

()

PSA

is the probability that the operator

lose adequate SA with a single vessel,

()

PSA

reflects the probability of single vessel obtain

adequate SA rely on various sensors. Conversely,

()

PSA is complementary set that shows probability

of not possessing adequate awareness. The operator

located on SCC receive operational data via console.

And the awareness of operator is reliant on

information transferred from the vessel via

communication channel

Com , however, the

communication link may be in available or

unavailable status.

()

PSA considering the

communication link can be described as



() ()

()( , )( ) ( , )( )

viv iv

PSA PSA

SA P SA SA Com P Com P SA SA Com P Com



 

, (6)

where

()PCom reflects the probability of

communication link is unavailable. According to the

characteristics of autonomous ships in “remote

control” mode, operator cannot obtain any awareness

when the communication link is valid, therefore,

(,)1

PSA SA Com  , and



() ()

()( , )( ) ( )

viv

PSA PSA

P SA P SA SA Com P Com P Com





. (7)

In this equation,

()

PSA can be described in

detail, which the vessel is equipped with a full set

equipment for navigation as main devices.

Additionally, the vessel needs to introduce the

redundancy to some safety-critical subsystems,

sensors or devices to mitigate many hazards and

make sure the navigation safety. Although such a

solution is said to be non-optimal, it is often named

as the first-choice-solution to ensure the adequacy of

control function. And the redundancy is proved

successful when ensuring the safety of complex

systems, which as a final line of defense against

critical devices' failure.

The probability of loss of awareness given by

vessel can be expressed as

()1 1 ( )

vjk

P SA P sensor







 







, (8)

where the number of safety-critical subsystems or

sensors is

l . For every subsystem or sensor, there is

one primary running device and

m redundancy.

Summing up, the proposed quantitative SA model as

follows:



() ( )(1 )

() () ()( )

()1 ()

() () ()( , )( ) ( )

()1 1 ( )

oii

ivviv

vjk

PSA PSA

PSA PSA PSA PSA

PSA PSA

SA P SA P SA P SA SA Com P Com P Com

P SA P sensor





















  









  







 













(9)

where



 

and

,,lmn  .

5 DISCUSSION

SA is crucial in maritime operations to identify

threats and to deal with them as soon as possible and

is an instrument for analysis of specific characteristics

and parameters of the monitored maritime object for

the purpose the obtained information about its

current status and forecasting its status in the near

future. The proposed SA quantitative model can

improve the accuracy of the collision risk

identification [30] and assessment, which can make

autonomous ships better comprehension of the

current situation.

In this paper, an applicable case should be

demonstrated to verify the validity of the quantitative

model. However, since the design and final structure

of autonomous ships is still being development, and

the actual operation has not yet occurred in reality. It

is temporarily impossible to obtain the data satisfying

the accuracy to calculate the probability of loss of SA.

At the same time, the model is constructed based on

the existing concept and composition system of

remotely controlled ship.

Therefore, it is necessary to make a lot of

assumptions of situation and probability as shown

the case in the paper. The assumptions based on the

theoretical basis or empirical formula is not adequate

sufficient, so the substantive value of the

demonstration is not significant. However, we still

make a numerical simulation in a specific context.

The same probability of loss of same sensors and

transmission link was assumed between the

autonomous ship and conventional ship. According

to characteristics of maneuvering, the crew control

directly vessel and perceive the situation of a ship on

the bridge. All ship-level information is shared with

the operator via the stable communication link.

Therefore, the probability that the operator lose

adequate SA on normal merchant ship can be

described

() ()

PSA PSA . (10)

According to the results of the calculation, the

failure probability of acquiring adequate SA of

remotely controlling vessels is significantly increased

compared to conventional vessels.

6 CONCLUSION AND OUTLOOK

In this paper, the concept of autonomous ships was

briefly discussed and defines the four development

stages under different control mode. According to the

initial literature review, the existing SA measurement

techniques provides a useful grounding in measuring

the elements that play a role in situational awareness,

but they cannot be applied to evaluate effectively the

SA of autonomous ships navigation, especially for the

remotely-controlled vessel.

On such a basis, the paper considers quantifying

the situational awareness of autonomous ships

navigation and proposed a model based on the

mathematical framework of Hierarchical Bayesian

Inference. The main result of numerical simulation

shows the autonomous ship' failure probability of

acquiring adequate SA is significantly higher than

conventional ship.

The significance of this paper is to present firstly a

quantitative processing of SA based on the system

safety control structure of autonomous ship in

“remote control” mode. In this model, more

important elements should be considered and

supplemented as the design and final structure

continue to improve. In addition, the model helpful

for detailed interface design and work domain

constraints in SCC and the futuristic concept of

autonomous unmanned shipping.

In future, we will study the obstacle avoidance in

navigation, where the autonomous ships can be

considered as intelligent agents or vehicles.

Therefore, we will investigate the routing algorithms

for both independent and cooperative agents (or

vehicles) in land and marine transportation [31-38], to

achieve the obstacle avoidance for autonomous ships.

ACKNOWLEDGMENT

This paper is partly supported by High-tech Ship Project

(80116003), Research on the Countermeasures of Maritime

Cooperation between China and ASEAN (80814011) and

China Association for Science and Technology.

REFERENCES

[1] H. Burmeister, W. Bruhn, Ø. J. Rødseth, and T. Porathe,

"Autonomous unmanned merchant vessel and its

contribution towards the e-Navigation implementation:

The MUNIN perspective," International Journal of e-

Navigation and Maritime Economy, vol. 1, pp. 1-13,

2014.

[2] R. Jalonen, R. Tuominen and M. Wahlström, "Safety

and security in autonomous shipping: challenges for

research and development," Remote and Autonomous

Ship-The next steps, pp. 56-72, 2016.

[3] International Maritime Organization (IMO), "Final

Report: Analysis of Regulatory Barriers to the use of

Autonomous Ships Submitted by Denmark," MSC

99/INF.3, London 2018.

[4] International Maritime Organization (IMO),

"Regulatory Scoping Exercise for The Use of Maritime

Autonomous Surface Ships (MASS)," MSC 99, London

2018.

[5] International Maritime Organization (IMO), "Report of

the Maritime Safety Committee on Its Ninety-Eighth

Session," MSC 98/23, London 2017.

[6] International Maritime Organization (IMO), "Maritime

Autonomous Surface Ships Proposal for a Regulatory

Scoping Exercise," MSC 98/20/2, London 2017.

[7] M. R. Endsley, "Design and evaluation for situation

awareness enhancement," in Proceedings of the Human

Factors Society annual meeting, 1988, pp. 97-101.

[8] K. Wróbel, J. Montewka and P. Kujala, "Towards the

development of a system-theoretic model for safety

assessment of autonomous merchant vessels,"

Reliability Engineering & System Safety, vol. 178, pp.

209-224, 2018.

[9] M. T. Dzindolet, S. A. Peterson, R. A. Pomranky, L. G.

Pierce, and H. P. Beck, "The role of trust in automation

reliance," International Journal of Human-Computer

Studies, vol. 58, pp. 697-718, 2003.

[10] R. Schönknecht, Ships and shipping of tomorrow:

Cornell Maritime Pr/Tidewater Pub, 1983.

[11] J. Barwise, The Situation in Logic: Conditionals and

conditional information vol. 2: Center for the Study of

Language and Information, Stanford University, 1984.

[12] J. Barwise, "Scenes and other situations," The journal of

Philosophy, vol. 78, pp. 369-397, 1981.

[13] J. Barwise, J. M. Gawron, G. Plotkin, and S. Tutiya,

Situation theory and its applications vol. 26: Center for

the Study of Language (CSLI), 1990.

[14] K. Devlin, "Situation theory and situation semantics,"

in Handbook of the History of Logic. vol. 7: Elsevier,

2006, pp. 601-664.

[15] V. Akman and M. Surav, "The use of situation theory in

context modeling," Computational intelligence, vol. 13,

pp. 427-438, 1997.

[16] M. R. Endsley, "Toward a theory of situation awareness

in dynamic systems," Human factors, vol. 37, pp. 32-64,

1995.

[17] G. Klein, "Cognitive task analysis of teams," Cognitive

task analysis, vol. 11, pp. 417-29, 2000.

[18] M. R. Endsley, "A survey of situation awareness

requirements in air-to-air combat fighters," The

International Journal of Aviation Psychology, vol. 3,

pp. 157-168, 1993.

[19] F. T. Durso, C. A. Hackworth, T. R. Truitt, J.

Crutchfield, D. Nikolic, and C. A. Manning, "Situation

awareness as a predictor of performance for en route

air traffic controllers," Air Traffic Control Quarterly,

vol. 6, pp. 1-20, 1998.

[20] R. M. Taylor, "Situational awareness rating technique

(SART): The development of a tool for aircrew systems

design," in Situational Awareness: Routledge, 2017, pp.

111-128.

[21] W. L. Waag and M. R. Houck, "Tools for assessing

situational awareness in an operational fighter

environment.," Aviation, space, and environmental

medicine, 1994.

[22] B. McGuinness and L. Foy, "A subjective measure of

SA: the Crew Awareness Rating Scale (CARS)," in

Proceedings of the first human performance, situation

awareness, and automation conference, Savannah,

Georgia, 2000.

[23] M. D. Matthews and S. A. Beal, "Assessing situation

awareness in field training exercises," Military

Academy West Point NY Office of Military Psychology

and Leadership 2002.

[24] K. Wróbel, J. Montewka and P. Kujala, "System-

theoretic approach to safety of remotely-controlled

merchant vessel," Ocean Engineering, vol. 152, pp. 334-

345, 2018.

[25] F. Sanfilippo, "A multi-sensor fusion framework for

improving situational awareness in demanding

maritime training," Reliability Engineering & System

Safety, vol. 161, pp. 12-24, 2017.

[26] M. Grabowski, A. Rowen and J. Rancy, "Evaluation of

wearable immersive augmented reality technology in

safety-critical systems," Safety science, vol. 103, pp. 23-

32, 2018.

[27] Y. Man, R. Weber, J. Cimbritz, M. Lundh, and S. N.

MacKinnon, "Human factor issues during remote ship

monitoring tasks: An ecological lesson for system

design in a distributed context," International Journal

of Industrial Ergonomics, vol. 68, pp. 231-244, 2018.

[28] J. De Winter, Y. B. Eisma, C. Cabrall, P. A. Hancock,

and N. A. Stanton, "Situation awareness based on eye

movements in relation to the task environment,"

Cognition, Technology & Work, pp. 1-13, 2018.

[29] International Maritime Organization (IMO). The IMO

regulatory framework and its application to Marine

Autonomous Systems. London: MSC 95/INF.20, 2015.

[30] X. Zhou, Z. Liu, F. Wang, and S. Ni, "Collision risk

identification of autonomous ships based on the

synergy ship domain," in 2018 Chinese Control and

Decision Conference (CCDC), 2018, pp. 6746-6752.

[31] Z. Cao, H. Guo, J. Zhang, F. Oliehoek and U.

Fastenrath, "Maximizing the Probability of Arriving on

Time: A Practical Q-Learning Method," in 31th AAAI

Conference on Artificial Intelligence (AAAI), pp.

4481-4487, 2017.

[32] Z. Cao, H. Guo, J. Zhang, D. Niyato, and U. Fastenrath,

"Improving the Efficiency of Stochastic Vehicle

Routing: A Partial Lagrange Multiplier Method," IEEE

Transactions on Vehicular Technology, vol. 65, no. 6,

pp. 3993-4005, 2016.

[33] Z. Cao, H. Guo, J. Zhang, D. Niyato, and U. Fastenrath,

"Finding the Shortest Path in Stochastic Vehicle

Routing: A Cardinality Minimization Approach," IEEE

Transactions on Intelligent Transportation Systems,

vol. 17, no. 6, pp. 1688-1702, 2016.

[34] Z. Cao, Y. Wu, A. Rao, F. Klanner, S. Erschen, W. Chen,

L. Zhang, and H. Guo, "An Accurate Solution To The

Cardinality-based Punctuality Problem," IEEE

Intelligent Transportation Systems Magazine, 2018.

[35] Z. Cao, H. Guo, and J. Zhang, "A Multiagent-Based

Approach for Vehicle Routing by Considering Both

Arriving on Time and Total Travel Time," ACM

Transactions on Intelligent Systems and Technology,

vol. 18, no. 3, Article No. 25, 2018.

[36] H. Guo, Z. Cao, M. Seshadri, J. Zhang, D. Niyato, and

U. Fastenrath, "Routing Multiple Vehicles

Cooperatively: Minimizing Road Network Breakdown

Probability," IEEE Transactions on Emerging Topics in

Computational Intelligence, vol. 1, no. 2, pp. 112-124,

2017.

[37] Z. Cao, H. Guo, J. Zhang and U. Fastenrath,

"Multiagent-based Route Guidance for Increasing the

Chance of Arrival on Time," in 30th AAAI Conference

on Artificial Intelligence (AAAI), pp. 3814-3820, 2016.

[38] Y. Cheng and W. Zhang, "Concise deep reinforcement

learning obstacle avoidance for underactuated

unmanned marine vessels," Neurocomputing, vol. 272,

pp. 63-73, 2018.