Towards Automated Performance Assessment for Maritime Navigation

ABSTRACT: This paper presents the outcome of a pre-project that resulted in an initial version (prototype) of an automated assessment algorithm for a specific maritime operation. The prototype is based on identified control requirements that human operators must meet to conduct safe navigation. Current assessment methods of navigation in simulators involve subject matter experts, whose evaluations unfortunately have some limitations related to reproducibility and consistency. Automated assessment algorithms may address these limitations. For a prototype, our algorithm had a large correlation with evaluations performed by subject matter experts in assessment of navigation routes. The results indicate that further research in automated assessment of maritime navigation has merit. The algorithm can be a stepping stone in developing a consistent, unbiased, and transparent assessment module for evaluating maritime navigation performance.

KEYWORDS: Ships Manoeuvering, Maritime Navigation, Automated Performance Assessment, Automated Assessment Algorithms, Save Navigation, Subject Matter Expert (SME) , Algorithms, Analytical Hierarchy Process (AHP)

REFERENCES

Allen, M. J. & Yen, W. M. (1979). Introduction to Measurement Theory. Belmont, CA: Wadsworth.

Bjørkli, C. A., Øvergård, K. I., Røed, B. K., & Hoff, T. (2007). Control Situations in High-Speed Craft Operation. Cognition, Technology, and Work, 9, 67-80. doi: 10.1007/s10111-006-0042-z

Bjørkli, C. A. & Øvergård, K. I. (2012). Automated assessment of docking maneuvers: When do we know when an operator performs well? Presentation at Scandinavian Maritime Conference 2012, 28-29 November at Vestfold University College, Horten, Norway.

Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The Dependability of Behavioral Measurements. London, England: John Wiley.

Danziger, S., Levav, J., & Avnaim-Pesso, L. (2011). Extraneous factors in judicial decisions. Proceedings of the National Academy of Sciences, 108(17), 6889-6892. doi: 10.1073/pnas.1018033108

Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors: The Journal of the Human Factors and Ergonomics Society, 37(1), 32-64. doi: 10.1518/001872095779049543

Flin, R. H., O'Connor, P., & Crichton, M. (2008). Safety at the sharp end: a guide to non-technical skills. Aldershot, England: Ashgate.

Freedman, D. A. (2009). Statistical Models: Theory and Practice, rev. ed. Cambridge, England: Cambridge University Press.

Fried, G. M., & Feldman, L. S. (2008). Objective assessment of technical performance. World Journal of Surgery, 32, 156-160. doi: 10.1007/s00268-007-9143-y

Gauss, B., & Kersandt, D. (2005). NARIDAS-Navigational Risk Detection and Assessment System for the Ship’s Bridge. In Proceedings of the International Conference on Computational Intelligence for Modelling, Control and Automation, 2005 (Vol. 2, pp. 612-617). IEEE.

Gauss, B., Rötting, M., & Kersandt, D. (2007). NARIDAS–evaluation of a risk assessment system for the ship’s bridge. In Human Factors in Ship Design, Safety and Operation. RINA-The Royal Institution of Naval Architects. International Conference.

Hederström, H., Kersandt, D., & Müller, B. (2012). Task-oriented structure of the navigation process and quality control of its properties by a nautical task management monitor (ntmm). European Journal of Navigation, 10(3).

Higgins, J. P. T. & Altman, D. G. (2008). Assessing risk of bias in included study. In J. P. T. Higgins and S. Green (eds.). Cochrane Handbook for Systematic Reviews of Interventions (pp. 187-242). West Sussex, England: John Wiley & Sons.

Higgins, J. P. T., Altman, D. G., Gøtzsche, P. C., Jüni, P., Moher, D., Oxman, A. D., Savovic, et al. (2011). The Cochrane collaboration's tool for assessing risk of bias in randomised trials. British Medical Journal, 343(7829), d5928. doi: 10.1136/bmj.d5928

Kahneman, D. (2011). Thinking, Fast and Slow. New York, NY: Farrar, Straus and Giroux.

Kahneman, D., Slovic, P., & Tversky, A. (1982). Judgment under uncertainty: Heuristics and biases. Cambridge, England: Cambridge University Press.

Kongsberg Maritime (2017) K-Sim Navigation – Kongsberg. Web site Kongsberg Maritime [Available at] https://www.kongsberg.com/en/kongsberg-digital/maritime%20simulation/k-sim%20navigation%20-page/

Manca, D., Nazir, S., Colombo, S., & Kluge, A. (2014). Procedure for automated assessment of industrial operators. Chemical Engineering Transactions, 36, 391-396. doi: 10.3303/CET1436066

Manca, D., & Brambilla, S. (2011). A methodology based on the Analytic Hierarchy Process for the quantitative assessment of emergency preparedness and response in road tunnels. Transport Policy, 18(5), 657-664. doi: 10.1016/j.tranpol.2010.12.003

Manca, D., Nazir, S., Lucernoni, F., & Colombo, S. (2012). Performance indicators for the assessment of industrial operator. Computer Aided Chemical Engineering, 30, 1422- 1426. Doi:10.1016/B978-0-444-59520-1.50143-3.

McCormack, W. (2007). Automated Operator and System Performance Assessment. In T. Bastiaens & S. Carliner (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2007 (pp. 7252-7259). Chesapeake, VA: Association for the Advancement of Computing in Education (AACE).

Nazir, S., Colombo, S., & Manca, D. (2013). Minimizing the risk in the process industry by using a plant simulator: a novel approach. Chemical Engineering Transactions, 32, 109-114. doi: 10.3303/ACOS1311028

Nazir, S., & Manca, D. (2015). How a plant simulator can improve industrial safety. Process Safety Progress, 34(3), 237-243. doi:10.1002/prs.11714

Nazir, S., Sorensen, L. J., Øvergård, K. I. & Manca, D. (2015). Impact of training methods on distributed situation awareness of industrial operators. Safety Science, 73, 136-145. doi: 10.1016/j.ssci.2014.11.015

Petersen, J. (2004). Control situations in supervisory control. Cognition, Technology, and Work, 6, 266-274. doi: 10.1007/s10111-004-0164-0

Saaty, T. L. (1980). The analytic hierarchy process: planning, priority setting, resources allocation. New York, NY: McGraw-Hill.

Saaty, T. L. (2007). Time dependent decision-making; dynamic priorities in the AHP/ANP: Generalizing from points to functions and from real to complex variables. Mathematical and Computer Modelling, 46(7), 860-891.

Øvergård, K. I. (2012). Absolute constraints, situation awareness and modelling of socio-technical systems. Presentation at the Scandinavian Maritime Conference 2012, 28-29 November at Vestfold University College, Horten, Norway.

Øvergård, K. I., Bjørkli, C. A., Røed, B. K. & Hoff, T. (2010). Control strategies used by experienced marine navigators: observations of verbal conversations during navigation training. Cognition, Technology, and Work, 12(3), 163-179. doi: 10.1007/s10111-009-0132-9

Øvergård, K. I., Nielsen, A. R., Nazir, S., & Sorensen, L. J. (2015). Assessing navigational teamwork through the situational correctness and relevance of communication. Procedia Manufacturing, 3, 2589-2596. doi: 10.1016/j.promfg.2015.07.579

Citation note:

Øvergård K.I., Nazir S., Solberg A.S.: Towards Automated Performance Assessment for Maritime Navigation. TransNav, the International Journal on Marine Navigation and Safety of Sea Transportation, Vol. 11, No. 2, doi:10.12716/1001.11.02.03, pp. 229-234, 2017

BibTeX EndNote

Authors in other databases:

Kjell Ivar Øvergård:

Salman Nazir: ORCID iD icon

orcid.org/0000-0002-2058-6147 Scopus icon