International Journal

on Marine Navigation

and Safety of Sea Transportation

Volume 3

Number 1

March 2009

Comparison of Traditional and Integrated

Bridge Design with SAGAT

F. Motz, E. Dalinger, H. Widdel & S. Hockel

Research Institute for Communication, Information Processing and Ergonomics,

Wachtberg, Germany

S. MacKinnon

Memorial University of Newfoundland, St. John’s, Canada

1 INTRODUCTION

Modern ship bridges are highly-automated man-

machine systems. Safety and efficiency of the ship

operations are dependent upon the ability of a

watchkeeper to perceive, interpret, and make deci-

sions upon information acquired from the surround-

ing environment. In the last years a strong increase

of modern information systems on ship bridges

could be observed. Simple displays and control sys-

tems were supplemented or replaced by complex

computer-based information systems. In order to

support the mariner effectively onboard, a task- and

situation-dependent representation of the informa-

tion is a compelling need. Modular Integrated Navi-

gation Systems (INS) according to the revised IMO

performance standards on INS (IMO 2007) combine

and integrate the validated information of different

sensors and functions and allow the presentation on

the various displays according to the tasks.

The aim of the investigations discussed within

this paper was to assess the advantages of an INS

design compared to a traditional bridge layout with

respect to the execution of collision avoidance and

route monitoring tasks. The Situation Awareness

Global Assessment Technique (SAGAT) (Endsley

2000) approach was applied to assess Situation

Awareness (SA) during these operations while

bridge design and out of window visibility were ma-

nipulated. The method to assess the SA of watch-

keeping officers on ship bridges was developed

based on previous studies (Motz et al. 2008).

The experiment was carried out in the full mis-

sion bridge simulator of the Centre for Marine Simu-

lation (CMS) of Memorial University of New-

foundland, St. John’s, Canada. The experimental tri-

als were conducted in a full bridge environment and

carried out with four scenarios, to investigate the in-

fluence of bridge design and outside visibility on the

SA of watchkeeping officers. The subjects were

tasked to navigate a vessel in scenarios with varying

traffic situations. In the trials a watch hand over was

simulated so that after the first 10 minutes of moni-

toring and evaluating the traffic situation the subject

assumed full control of the vessel.

ABSTRACT: Modular Integrated Navigation Systems (INS) according to the revised IMO performance stan-

dards on INS combine and integrate the validated information of different sensors and functions and allow the

presentation on the various displays according to the tasks. The aim of the investigations discussed within this

paper was to assess the advantages of an INS design compared to a traditional bridge layout with respect to

the execution of collision avoidance and route monitoring tasks. The Situation Awareness Global Assessment

Technique (SAGAT) approach was applied to assess Situation Awareness (SA) during these operations while

bridge design and out of window visibility were manipulated. Additionally, workload was measured by

NASA-TLX with its six subscales. The experiment was conducted in a full immersive simulation environ-

ment with 26 experienced mariners. The results indicate that SA is significantly higher with the INS bridge

design in the reduced visibility condition compared to the traditional bridge design. Also, tendencies were

found that workload and subdimensions are influenced by bridge design and visibility conditions.

2 SIMULATOR AND BRIDGE DESIGN

The experiment was carried out in a 6 degree of

freedom motion base, full mission ships bridge sim-

ulator (see Fig. 1). The simulator was manufactured

by Kongsberg Maritime Ship System. All trials were

conducted under a repeated motion profile. In the

experiment two bridge configurations were com-

pared: a traditional layout employing the existing

navigational equipment and an INS design. The ma-

jor difference between the two designs is the factor

of integration of the collision avoidance and naviga-

tion information, including route monitoring and

planning. The arrangement, location and design of

the equipment of both configurations were identical.

Figure 1. Full mission ships bridge simulator.

The following set up was used as traditional con-

figuration:

− radar with facilities to display tracked radar tar-

gets

− electronic Charting Display and Information Sys-

tem (ECDIS) with route information

− minimum keyboard Display (MKD) to display

AIS target information

− depth sounding information

− heading information

− speed information

− VHF communication (Navtex, VHF DSC)

− propulsion status displays

− steering and engine control

− steering status displays

− alarm information presented on the individual

equipment

− chart table with paper charts.

For the INS configuration the information of the

various navigational systems were integrated and

combined in the displays on the bridge:

− collision avoidance display: radar with tracked

radar targets and AIS targets, possibility to under-

lay ENC chart information

− route monitoring display: ECDIS with active

route and AIS targets

− conning display: position, propulsion informa-

tion, rate of turn, relative wind speed and direc-

tion, engine alarm status, rudder indicator, gyro

repeater and speed

− minimum keyboard Display (MKD) to display

AIS target information

− speed information

− VHF communication (Navtex, VHF DSC)

− propulsion status displays

− steering and engine control

− steering status displays

− alarm information presented on the individual

equipment

− chart table with paper charts.

3 SAGAT

Situation Awareness is generally understood as

"knowing what is going around you". Within the re-

search community the definition of Endsley (1995)

has been widely accepted in various domains. In a

cognition-oriented approach, the model considers

three levels and includes:

− perception of elements,

− comprehension of the meaning of the elements

and the situation, and the

− projection of the status of the elements and the

situation into the immediate future.

According to this model, decision making and

performance is influenced by SA.

The Situation Awareness Global Assessment

Technique (SAGAT) is probably one of the most

well known SA measuring techniques (Endsley

2000). It provides an objective explicit measure of

SA by directly comparing the operator’s SA to an

operational “scenario”. With this technique, a simu-

lation is frozen at a specific time, the system dis-

plays are blanked while the operator quickly has to

answer questions concerning the scenario. Tempo-

rary freezes in the simulation must be of a short du-

ration to minimise intrusiveness and memory decay.

To get an accurate measure of the operator’s SA the

SAGAT probes must cover all three levels of SA

and must be reflective of a wide range of the SA re-

quirements. These are delineated through a goal-

directed task analysis.

Table 1. Samples of SA questions.

Level Question

Perception

What is the current position of your ves-

sel?

What is the course of the blue highlighted

vessel?

Comprehension

What is the distance to the next waypoint?

What is the direction of the course change

of the highlighted vessel?

Projection

What is the CPA of the blue highlighted

vessel?

In how many minutes will you reach the

pilot station?

The method originally was developed for the avi-

ation domain (e.g. Endsley 1990, Strater & Endsley

2000) and has been applied in identical or modified

forms in other domains. Presently, it was adapted to

the marine domain (Motz et al. 2008). A sample of

SAGAT probes for marine application is shown in

Table 1.

SAGAT can be a useful tool to evaluate system

design. SAGAT can provide a form of diagnostic in-

formation that indicates how a technology’s design

could improve or weaken an operator’s SA when

compared to a baseline technology. This information

can then be used to refine design concepts. For the

ergonomic evaluation of the task- and situation ori-

entated presentation of navigational information on

INS it was considered to use the concept of situation

awareness measured with SAGAT.

Questions related to perception of elements (level

1) refer to the status of own ship as well as dynamics

of relevant objects in the environment. A mariner

has to possess correct information of own ship (posi-

tion, route, course etc.) as well as correct infor-

mation about targets (speed, distance etc.).

Questions related to comprehension of meaning

(level 2) go beyond simply being aware of the ele-

ments that are presented. An understanding of the

significance of those elements in light of mariner’s

goals is included. For example, a mariner must

quickly determine those targets which pose a threat

and eventually demand operator action to mediate a

threat or obstacle.

Questions related to projection of the near future

(level 3) refer to future actions of the elements in the

environment. This is achieved through knowledge of

the status and dynamic of the elements and a com-

prehension of the situation.

4 EXPERIMENT

4.1 Subjects

26 experienced mariners (masters, navigational offi-

cers, pilots) participated in the trials. The require-

ments for subject recruitment were:

− at least half a year experience as officer of the

watch (OOW)

− the mariners must have navigated a vessel in the

last four years

− or working actively as navigational simulator in-

structors.

4.2 Hypotheses

It was hypothesized that navigating with INS bridge

design leads to higher SA scores than navigating

with traditional bridge design. This effect might be

more distinctive under difficult conditions like re-

duced outside view when fog prevails than under

good view. In the case of bad navigation conditions

also mental workload may be increased.

4.3 Experimental Design

A 2 x 2 factorial design with two within-subject fac-

tors was used. The first within-subject factor bridge

design varied on the two levels:

− traditional configuration

− INS configuration.

− The second within-subject factor visibility varied

on the two levels:

− good visibility

− reduced visibility (fog).

Dependent variables were SA and NASA-TLX

scores (Hart & Staveland 1988).

A 2 x 2 repeated measures ANOVA (Analysis of

Variance) model with the within subject factors

bridge design and visibility were employed to ana-

lyse these data. As dependent variables the relative

frequency of correct answers representing the SA

score were analyzed. For workload the NASA-TLX

scores were used. All statistical interpretations were

considered at the 5 % level of significance.

4.4 Experimental Control

The experiment was carried out in the ships bridge

simulator with the two bridge designs described in

Section 2.

To control the presentation of SA questions on

the bridge, to log the answers of the subjects and to

monitor the vessels involved in a scenario an exper-

imental controlling program was developed by Re-

search Institute for Communication, Information

Processing and Ergonomics. The program consists

of the question presentation module, the nautical

chart module presenting the graphical information

for the SA questions and the experimental control

module (see Fig. 2). The three different modules

were located on different computers in the network.

Whereas the nautical chart module and the question

presentation module were installed on the naviga-

tional bridge of the simulator, the experimental con-

trol module runs on a PC in the simulator control

room.

Figure 2. Controlling components of experiment.

The experimental control module fulfilled the

three main tasks processing of simulator output, tar-

get control and control of SA questioning.

The experimental control module read and pro-

cessed the data of own ship’s GPS output signal and

the AIS signals of the other vessels provided to the

navigational equipment (e.g., the radar, ECDIS or

the MKD) on the bridge. This allowed the SA ques-

tions to be posed using real-time data. Therefore, the

software offered the possibility to show during the

simulation the movement of own ships and the other

vessels on a chart background and to offer sugges-

tions to change the course or speed of the vessels in

case of course or speed alterations of the own ship in

order to present the SA questions in similar traffic

situations for all participants.

4.5 Presentation of SA questions on the bridge

When the scenarios were “frozen” to present the

subjects the SA questions, the simulation was sus-

pended, all task relevant information was removed

from the navigational equipment on the bridge and

the outside view was blanked. The SA questions

were then administered on two displays on the

bridge. Whereas on one of the displays the questions

were presented, on the other display additional

graphical information for certain questions were dis-

played (see Fig. 2).

This offered the four techniques to ask the SA

questions, depending on the topic, complexity and

the easiest way to present and to answer the ques-

tions:

− numeric open-end questions: Questions appeared

on the question display as numeric open-end

questions, e.g., what is your heading after the

next waypoint. No graphical information was pre-

sented on the graphical information display. To

answer the questions the subjects had to type in

the right answer and then to click on the continue

button to proceed with the next question

− open-end questions with graphical answer: The

question after the position of the vessel appeared

on the question display as open-end question (in-

struction) and had to be answered on the graph-

ical information display by clicking on the chart

background.

− numeric open-end questions with additional

graphical information: Questions appeared on the

question display as open-end questions with addi-

tional information presented on the chart of the

graphical information display, e.g., the target in-

volved in the question. To answer the questions

the subjects had to type in the right answer and

then click on the continue button to proceed with

the next question

− multiple choice questions with additional graph-

ical information: Questions appeared on the ques-

tion display as multiple choice questions and on

the graphical information display question related

information was presented on the chart back-

ground. To answer the questions the subjects had

to select the right answer and then to click on the

continue button to proceed with the next question.

A preliminary question-pool of about 70 ques-

tions had been developed referring to the navigation

of the vessel focusing on route monitoring and colli-

sion avoidance. The questions were evaluated in pre-

tests which had the aim of selecting the most rele-

vant questions, of testing the content and under-

standability, and to reduce the number of questions

resulting in a set of 16 questions on three levels (see

Table 1 for examples).

4.6 Traffic scenarios

The SAGAT approach required the development of

realistic scenarios based on specific criteria, e.g.,

course change of own ship, navigational hazards,

traffic density, and “interest/danger” of targets. The

criteria for traffic density and “interest” of targets

are:

− total number of targets - 20

− number of targets within a 3 NM range - 10

− number of targets within a 3 NM range: close

quarter targets (CPA: 0.5 - 1.5 NM); with a colli-

sion course, overtaking own ship or overtaken by

own ship - 5

− number of targets, which cause a reaction because

of collision course - 1.

Figure 3: Singapore scenario at the first freezing.

On the basis of these criteria four traffic scenarios

with duration of 21.5 - 25 minutes were developed,

as well as one familiarization/habituation scenario.

To ensure that the previous knowledge of traffic and

of the sea area doesn’t influence the results (i.e. a

learning effect was controlled for), the scenarios rep-

resent different traffic situations for the Juan de Fuca

Strait / Strait of Georgia (familiarization scenario),

English Channel and Singapore. Figure 3 shows one

Singapore scenario at the time of the first freezing.

In the pre-trials the traffic scenarios were evaluated

in respect to realism, relevance and complexity.

4.7 Experimental procedure

The experimental procedure had five steps.

In the first step the experiment was introduced to

the subjects in a briefing outside the simulator. Sub-

jects had to complete a personal data sheet, which

was used to gather data like current occupation,

years of experience as mariner, and age, and the in-

tention of the experiment and general description of

experimental set up were described to the subjects.

In the second step subjects were familiarized in

the simulator with the experimental procedure and

the bridge equipment. The INS design and the tradi-

tional layout were explained in detail and the proce-

dure with the presentation of scenarios and the fol-

lowing interruptions for SA questioning were

explained.

In the third step familiarization trials were con-

ducted, one familiarization trial for each bridge lay-

out. The purpose was to familiarize the subject with

the navigation of the vessel with the different bridge

layouts, with the experimental procedure of the

freezings and with the different types of SA ques-

tions. The familiarization trials were carried out

without motion and with good visibility for all trials.

In the next step the four scenarios were presented

to the subject in a randomized order. The task of the

subject was to navigate a vessel in traffic situations

of varying density with either good or reduced visi-

bility. In the trials a watch hand over was simulated

so that the first 10 minutes of each scenario the sub-

ject monitored and evaluated the traffic situation. An

instructor was fulfilling the role of the officer of the

watch for the first 10 minutes. After the hand over,

the subject was in full control of the vessel. An in-

structor remained on the bridge and acted as both the

helmsman and the master. Thus, as the helmsman,

the instructor performed any changes in speed and

course and as master, to deny inappropriate deci-

sions of the subject that might disrupt the whole ex-

periment.

During each scenario there happened four freez-

ings in which the outside view and the displays were

blanked and the SA questions were asked. The first

freezing was conducted at the watch hand over and

the last at the end of the scenario. Same questions

were asked for all treatments, 16 questions per sce-

nario divided into 4 groups of 4 questions.

At the end of each scenario the NASA-TLX rat-

ing scale was completed. Following the collection of

all four scenarios the NASA-TLX rating paired

comparisons questionnaire were completed.

The duration of a simulation trial (4 scenarios and

habituation) per subject was between 190 to 220

minutes, depending on the time each subject needed

to become familiarize with the bridge equipment.

The trials were carried out with motion.

After the trials a SAGAT debriefing question-

naire and a INS questionnaire to evaluate certain as-

pects of an INS layout were administered to the sub-

jects in a separate room.

5 RESULTS

5.1 Main results

The central questions of the experiment were fo-

cused on the impact of the independent variables,

bridge design and visibility, on situation awareness.

The means of frequencies of correctly answered SA

questions are shown in Figure 4. The results of the

ANOVA for the within subject factors bridge design

and visibility show a significant main effect for the

factor bridge design (F

1,25

= 4.88, p < 0.05) and a

significant interaction effect between bridge design

and visibility (F

1,25

= 6.94, p < 0.05). No significant

effect for the factor visibility (F

1,25

= 0.94, p > 0.3)

could be found.

Good Visibility Reduced Visibility

Means of correct answers [%]

trad. bridge

INS

Figure 4. Dependency of SA from bridge design and visibility.

The analysis of variance executed for the overall

workload, defined by the total score of NASA-TLX,

does not result in significant differences for the two

main effects visibility (F

1,25

= 3.57, p = 0.07) and

bridge design (F

1,25

=1.01, p=0.32) or for the interac-

tion (F

1,25

= 0.13, p = 0.72). But a strong tendency

can be seen that the INS produces less workload

than a traditionally designed bridge, and reduced

visibility is responsible for higher workload. Alt-

hough, tendencies were found for the subscales per-

formance and effort favoring the INS bridge design

especially under the condition of reduced visibility.

Results from the INS questionnaire support the

empirical data collected in this study. In general, the

majority of the subjects (93%) preferred the INS

bridge design compared to traditional design. The

participants who preferred the traditional bridge

gave as reasons that they are more used to the tradi-

tional design and that the INS design provides too

much information. As added value of an INS almost

all participants chose the answer “the combination of

information” (see Fig. 5). Half of the respondents se-

lected “the integrity of data” (meaning the possibil-

ity to compare automatically data from independent

sources). Less often selected is “the higher quality of

information”.

100

Combination Higher quality Integrity Other

Frequency of answers [%]

Figure 5. Means of frequency for the added value of an INS

bridge.

5.2 Post-hoc analysis

The definition of SA specifies a hierarchical struc-

ture with three levels (see Section 3). Questions for

the first level are preconditions for answering ques-

tions on level 2 and 3. Following this hierarchical

organization leads to the assumption that questions

of level 1 are answered correctly more often than

questions of the higher levels.

In the post-hoc analysis the factor level of SA was

included. A 2 x 2 x 3 ANOVA with the factors

bridge design, visibility and level of SA as within

subject factors was performed to test the assumption

which was justified by a significant main effect of

the factor level of SA (F

2,50

= 43.47, p < 0.001).

A pairwise comparisons with Bonferroni correc-

tion of the three SA level scores show a significant

higher score for level 1 compared to level 2 and lev-

el 3, but no difference between the latter two. In

Figure 6 the means of frequencies of correct answers

for the 3 levels of SA are shown for condition re-

duced visibility.

SA Level 1 SA Level 2 SA Level 3

Means of correct answers [%]

trad. bridge

INS

Figure 6. Dependency of SA from bridge design on the three

SA levels.

SA level 1 (perception) had a greater score than

SA levels 2 and 3 (comprehension and projection),

suggesting that the application of SAGAT in this

maritime-related research was a valid approach to

assess global SA.

6 CONCLUSIONS

The results indicate that bridge design has a signifi-

cant impact on the degree of situation awareness, as

hypothesized. SA is significantly higher with the

INS bridge design in the reduced visibility condition

compared to the traditional bridge design. In good

visibility the SA is similar with both bridge designs.

Mariners have to rely more on information systems

when navigating in reduced visibility conditions. It

can be hypothesized that not only reduced visibility

but detrimental navigational conditions, in general,

may reduce SA when navigating with traditionally

designed bridges but not with INS. These considera-

tions also apply to workload in the sense that stress

inducing work conditions can influence total work-

load and subdimensions of NASA-TLX like perfor-

mance and effort when using traditional bridge de-

sign.

Further experiments are required comprising

more difficult navigation surroundings, e.g., higher

traffic density, more challenging navigation tasks,

high stress inducing work environment, to confirm

and sharpen these experimental findings.

REFERENCES

Endsley, M.R. 1990. A methodology for the objective meas-

urement of situation awareness. In Situational Awareness in

Aerospace Operations (AGARD-CP-478): 1-9. Neuilly sur

Seine, France: NATO- AGARD.

Endsley, M.R. 1995. Toward a theory of situation awareness in

dynamic systems. Human Factors 37: 32-64.

Endsley, M.R. 2000. Direct Measurement of Situation Aware-

ness. In M.R. Endsley & D.J. Garland (eds.), Situation

Awareness Analysis and Measurement: 147-173. Mahwah,

NJ: Lawrence Erlbaum Associates.

Hart, S.G. & Staveland, L.E. 1988. Development of NASA-

TLX (Task Load Index): Results of empirical and theoreti-

cal research. In P. Hancock & N. Meshkati (eds.), Human

Mental Workload: 139-183. Amsterdam: North Holland.

IMO 2007. Revised performance standards for integrated navi-

gation systems (INS). MSC.252(83). London: International

Maritime Organization.

Motz, F., MacKinnon, S., Dalinger, E., Widdel, H. 2008. Ap-

plication of SAGAT for Navigational Tasks on Ship Bridg-

es. International Maritime Conference / ABCD Symposium

on Human Performance at Sea. Sydney, Australia.

Strater, L. & Endsley, M.R. 2000. SAGAT- A situation aware-

ness measurement tool for commercial airline pilots. In

D.B. Kaber & M.R. Endsley (eds.), Proceedings of the first

human performance, situation awareness, and automation:

User-centered design for the new millennium conference:

360. Savannah, GA.