International Journal

on Marine Navigation

and Safety of Sea Transportation

Volume 4

Number 1

March 2010

1 INTRODUCTION

A decrease in crew performance for maritime works

can be caused by the complex causation related to

physiological, psychological, and external sailing

factors (Kim et al., 2004). A procedure in maritime

accidents caused by the reduction of crew perfor-

mance can be explained as follows. Physiological,

psychological, and external sailing factors affect the

working process of a marine pilot directly or indi-

rectly. These factors decrease physical and psycho-

logical abilities and that ultimately affect decreases

in the cognitive performance of crews as ultimate

factors. The decrease in cognitive performance caus-

es mistakes, such as negligence of lookout, and that

lead to a direct cause of accidents.

As shown in Figure 1, human cognitive perfor-

mance represents all abilities of the elements pre-

sented in an information processing model of human

(Wickens, 1992). However, it may not be necessary

to measure the all abilities of such cognitive ele-

ments in a project that investigates the cognitive per-

formance of a worker who processes given works. In

general, there exist cognitive elements to play a de-

finitive role in the effective performance for given

cognitive works. Because these elements are enough

to perform such given works except for a decrease in

cognitive performance caused by certain diseases, it

is possible to estimate the cognitive performance of

a worker in given works using such definitive ele-

ments in the cognitive works.

Figure 1. Model of human information processing

Subjective methods, physiological monitoring,

and task loading methods are generally used to eval-

uate these cognitive performances. Also, these

methods have been applied to some high risk indus-

tries, such as national defense, road transportation,

railways, aerospace, process control, and power gen-

eration in which the selection of a method usually

depends on specific requirements related to each in-

dustry.

A representative study in subjective methods is

the Modified Cooper-Harper Scale (MCH) (Wier-

wille and Casalli, 1983) that complemented the

Collision Scenario-based Cognitive

Performance Assessment for Marine Officers

H. Kim & H.-J. Kim

Marine Transportation & Pollution Response Research Dept., MOERI/KORDI, Daejeon,

Korea

S. Hong

Chungju National University, Chungju, Korea

ABSTRACT: The overall aim of this paper is to determine a fatigue factor that can be applied to human per-

formance data as a part of a software program that calculates total cognitive performance. This program ena-

bles us to establish the levels of cognitive performance in a group of marine pilots in order to test a decision-

making task based on radar information. This paper addresses one of the factors that may contribute to the de-

termination of various fatigue factors: the effects of different work patterns on the cognitive performance of a

marine pilot.

Cooper-Harper Scale, which was developed to eval-

uate the performance of the handling characteristics

of military aircrafts in the end of the 1960s. In addi-

tion, the NASA TLX (Hart and Staveland, 1988) is a

biploar-rating scale-based study using self-report

scores.

In the case of the physiological monitoring, there

are some studies on the variation of human physiol-

ogy responses, such as Electroencephalogram

(EEG), Electrocardiogram (ECG), Electrodermal ac-

tivity (EDA), and Electrooculogram (EOG), accord-

ing to task demands (Andreassi, 2000).

The task loading methods represent an engineer-

ing approach that is to measure workloads based on

the estimation of task demands. The Task Analysis

Workload (TAWL) Methodology (Mitchell, 2000)

that was developed by using the US Army Light

Helicopter Experimental Program and the Operator

Function Model-Cognitive Task Analysis (OFM-

COG) (Lee and Sanquist, 2000) that was developed

to evaluate workloads in ship-borne automation sys-

tems applied these methods.

Marine officers perform various cognitive works,

such as signal detection, situation recognition, gen-

eral judgment, and other related works, in their ship

operation jobs. For instance, it can be considered as

perceptual ability to recognize target ships ap-

proached to their own ship through radars and the

naked eye, memory ability in a steersman who

memories the commands from his captain, and

judgment ability to determine the scale of the con-

version (heading) of the bow to avoid the collision

with approached target ships. It is difficult to guar-

antee that such cognitive works occur intermittently

or sequentially. Requirements in excessive cognitive

performance may cause some mistakes in marine of-

ficers and that lead to maritime accidents (Lee,

2005). However, there are still limited studies on the

quantitative evaluation of the cognitive performance

for maritime officers.

Thus this study developed a maritime collision

scenario-based cognitive performance evaluation

system for marine officers. The evaluation criteria

was configured by applying practical experiments

for a group of marine pilots and verified the system

through practical applications for cadet marine pi-

lots. Because this system is able to evaluate general

cognitive performance of marine officers, it is able

to play a role in the avoidance of accidents based on

their own awareness on such accidents by transfer-

ring the results of the evaluation of physical and

psychological conditions through applying a test for

a short period of time before going on duty or board-

ing.

2 COLLISION SCENARIO-BASED COGNITIVE

PERFORMANCE ASSESSMENT

In this study, we developed a computer program to

evaluate the abilities of signal detection and deci-

sion-making task in cognitive performance for ma-

rine officers. The objective of this program is to

measure the perceptual ability (signal detection) of

marine officers for searching other ships through the

information presented on radars and the judgment

ability (situation recognition or decision-making)

that determines the direction and speed of a ship to

avoid the collision with other ships. The cognitive

performance evaluation program for marine officers

developed in this study reflects general cognitive

abilities for operating a ship and measures the per-

formance through a 10 minute simple test before go-

ing on duty or boarding.

Also, this system is a program that measures the

cognitive performance of a marine pilot who con-

trols the heading and speed of a ship using the in-

formation presented in a ship operation process. In

general, the information given to marine officers is

the data presented on radars and speed information

of their own ship. The marine officers possibly ob-

serve a planned course and control the heading and

speed of their own ship in order to avoid the colli-

sion with other ships. After avoiding possible colli-

sion, the marine officers should return its own

course.

Figure 2. Screen of an evaluation of cognitive performance

Figure 2 illustrates a screen image of the cogni-

tive performance evaluation. The left side of the

screen represents the information of target ships

(DCPA, TCPA, Heading, Speed, Bearing, and

Range) and the right side shows the input menu of

the information for changing a course. Whereas, the

DCPA (Distance at Closest Point of Approach)

shows the estimated distance to the recent closest

point and the TCPA (Time to Closest Point of Ap-

proach) demonstrates the estimated time to the re-

cent closest point. In order to attempt a proper action

for collision avoidance, it is necessary to input the

action time for collision avoidance, heading and

speed at the starting point, termination time for colli-

sion avoidance, and heading and speed at the termi-

nation point.

This system configured 10 collision scenarios as

noted in Table 1 by varying the number of target

ships, heading and speed, and bearing based on the

four rules presented in the International Regulations

for Preventing Collisions at Sea (1972).

Table 1. Types of scenarios

Although the Scenarios 1 and 3 show the same

situation, “Head-on Situation”, target ships represent

different headings and speeds. The Scenarios 2 and 5

show the same situation, “Crossing Situation”, but

they represent different numbers of target ships,

such as one and two ships. Also, the Scenarios 4 and

6 show the same situation, “Overtaking”, but they

demonstrate different headings and speeds.

This system configured a scoring index to evalu-

ate the cognitive performance of marine officers as

follows.

1 Collision avoidance ability

2 Decision-making time

3 Degree of deviation

The evaluation criteria were produced for each

scenario with advice from professional marine pi-

lots. They were not participated in the experiments.

Table 2 shows the evaluation criteria for “Crossing

Situation”.

3 EXPERIMENTS & RESULTS

Three professional marine pilots and five cadet ma-

rine pilots were participated to verify the evaluation

of the cognitive performance assessment system for

marine officers developed in this study. Except for

Scenario 1, which was applied as a pretest, experi-

ments were applied to other nine Scenarios.

Figure 3 shows a screen image of the collision

scenario of “Traffic Separate Schemes". An experi-

ment based on this scenario represents the input and

analysis data as shown in Figure 4.

Table 2. Evaluation criteria for “Rule 15 "

Figure 3. Screen of the scenario of “Rule 10 (TSC)”

Figure 4. Screen of the test results

In the results of these experiments, the profes-

sional pilots showed higher scores than cadet marine

pilots, average 90.2 and 74.0.

Also, as shown in Figure 5, the total scores of the

professional pilots for scenarios showed high levels

more than 10 points compared to that of the cadet pi-

lots.

Figure 5. Comparison of the total scores by scenarios

For a comparison and analysis of this data, a 5%

level of significance paired-wised t-test was con-

ducted. According to the analysis result, there was a

statistically significant difference in the total scores

between the professional pilots and the cadet pilots

for each experiment subject (p=0.015).

In addition, in the results of the comparisons of

the Distance to the Closest Point of Approach

(DCPA) that is the most important factor to achieve

collision avoidance, the professional pilots showed

higher scores than cadet pilots for all scenarios as il-

lustrated in Figure 6 in the “Scoring Index”.

For a comparison and analysis of this data, a 5%

level of significance paired-wised t-test was con-

ducted. According to the analysis result, there was a

statistically significant difference in the DCPA

scores between the professional pilots and the cadet

pilots for each experiment subject (p=0.028).

Figure 6. Comparison of the DCPA scores by scenarios

Regarding future studies, it will attempt to guar-

antee the evaluation data through additional experi-

ments in order to complement the cognitive perfor-

mance evaluation system for marine officers and that

will increase the reliability of this system.

4 CONCLUSIONS

In recent years, various sailing equipments, such as

GPS, ARPA, ECDIS, AIS, VDR, and hull monitor-

ing system, have been introduced to ship operation

and the development of such hardware still have

been conducted.

However, the improvement and effort on the ship

operator-based related software are still limited and

in an elementary step.

The present circumstance is due to the lack of in-

vestment in this filed even though there are some

Total Scores

Expert

Cadet

words on the marine accidents that usually caused

by human factors. It can be considered that there are

still lack of studies on physical, psychological, and

cognitive performance for marine officers who guar-

antee the safety of sailing using advanced equip-

ments and consideration.

This study attempted to develop a cognitive per-

formance assessment system for marine officers that

evaluates the cognitive performance of marine offic-

ers through a simple way before going on duty or

boarding and provides the results of the evaluation

to the pilot as a warning message for avoiding ma-

rine accidents caused by the decrease in cognitive

performance of marine officers.

In addition, there exist some problems for the re-

flection of the importance in detailed items that con-

sist of the reflection issues of difficulties and evalua-

tion criteria according to collision scenarios in the

experiment and analysis processes in this study.

The result of the analysis in this study includes

some problems of the limited subjects and quantita-

tive evaluation criteria. Also, the cognitive perfor-

mance assessment system developed in this study

included the evaluation of the cognitive performance

only, future studies will reflect an evaluation model

for the fatigue of marine officers by considering

their sleep conditions and workloads and establish a

reasonable and reliable evaluation system by accu-

mulating various collision scenarios and by com-

plementing the existing evaluation criteria.

ACKNOWLEDGEMENTS

The authors would like to special thank Professor

Mike Barnett and the instructors/cadets from

Warsash Maritime Academy for their support with

the research. The contents of this paper are the re-

sults of the research project of MOERI/KORDI

(Analysis of Tug-Barge Accident and its Prevention)

REFERENCES

(1) Andreassi, J.L., 2000. Psychophysiology: Human behaviour

and physiological response (4th edition), Lawrence Erl-

baum Associates.

(2) Kim, H., Yang, C.S., Park, J.H. and Lee, J.K., 2004. Reduc-

tion of maritime risk relating human factors, Proceedings of

the Society of Naval Architecture of Korea(Spring Confer-

ence), pp. 816-821.

(3) Hart, S.G. and Staveland, L.E., 1988. Development of

NASA-TLX (Task Load Index): results of empirical and

theoretical research in: Hancock, P.A. and Meshkati,

N.(eds.), Human Mental Workload. Amsterdam: North-

Holland.

(4) Mitchell, D.K., 2000. Mental Workload and ARL Work-

load Modeling Tools, Army Research Laboratory, Report

No ARL-TN-161.

(5) Lee, J.D., and Sanquist, T.F., 2000. Augmenting the opera-

tor function model with cognitive operations: assessing the

cognitive demands of technological innovation in ship nav-

igation, IEEE Transactions on Systems, Man and Cybernet-

ics – Part A: Systems and Humans. 30 (3), 273-285.

(6) Lee, J.K., 2005. Development of Fundamental Technolo-

gies for Total Risk Management, KORDI Project Report,

UCE00940-05043.

(7) Wickens, C.D., 1992. Virtual reality and education, Pro-

ceedings of the 1992 IEEE International Conference on

Systems, Man and Cybernetics, 1, 842-847.

(8) Wierwille, W.W. and Casali, J.G., 1983. A validated rating

scale for global mental workload measurement application,

Proceedings of the Human Factors Society 27th Annual

Meeting. Santa Monica, CA: Human Factors Society.