269
1 INTRODUCTION
Consideringtheglobalmaritimecyberrisklandscape,
the likelihood of maritime digital systems becoming
thetarget ofacyberincidenthasincreasedinrecent
years [1]. Research indicates that critical onboard
systems are susceptible to compromise by both
accidental actions and deliberate interference [2].
There are currently several
approaches to managing
thesethreats.Firstly,theUNSpecialised Agencythe
International Maritime Organization (IMO) has
provided highlevel requirements and
recommendationsforcybersecurityonboardships[3,
4]. Secondly, one of the largest global shipping
associations BIMCO has provided a maritime cyber
risk managementspecific framework for preparing
againstthecyberthreatonanorganisationallevel[5].
Thirdly,theInternationalAssociationofClassification
Societies (IACS), has recently published two new
Unified Requirements (UR) considering cyber
resilience for ships, namelyʺE26 Cyber resilience of
shipsʺandʺE27Cyberresilienceofonboardsystems
andequipmentʺ.AsIACSconsistof
thelargest class
societies in the world, covering a majority of the
worldʹsfleet,theseURswillhaveaworldwideimpact
[6]. However, with these requirements only being
implemented on new builds from 1st January 2024,
the realisation of these impacts will be a long time
coming.
All the
above documentation is designed to aid
shipownercompaniesinthemanagementoftherisks
theyfaceduetoconnectedtechnology.However,on
board ships, the cyber risks are still being handled
pragmatically and by improvisation, as seafarers
currentlyhavelittletonoformalizededucationabout
thecyberriskstheyface[7].
Thus,thereisaneedfor
operational tools which can be used by the crew in
CERP: A Maritime Cyber Risk Decision Making Tool
E.Erstad
1
,R.Hopcraft
2
,J.D.Palbar
2
&K.Tam
2
1
NorwegianUniversityofScienceandTechnology,Ålesund,Norway
2
UniversityofPlymouth,Plymouth,UnitedKingdom
ABSTRACT:Anincreaseinthecomplexityofsystemsonboardshipsinthelastdecadehasseena riseinthe
numberofreportedmaritimecyberattacks.TotacklethisrisingrisktheInternationalMaritimeOrganization
published highlevel requirements for cyber risk management in
2017. These requirements obligate
organisationstoestablishprocedures,likeincidentresponseplans,tomanagecyberincidents.However,there
is currently no standardised framework for this implementation. This paper proposes a Cyber Emergency
Response Procedure (CERP), that provides a framework for organisations to better facilitate their crew’s
response to a cyberincident
that is considerate of their operational environment. Based on an operations
flowchart,theCERPprovidesastepbystepprocedurethatguidesacrew’sdecisionmakingprocessintheface
of a cyberincident. This highlevel framework provides a blueprint for organisations to develop their own
cyberincidentresponse
proceduresthatareconsiderateofoperationalconstraints,existingincidentprocedures
andthecomplexityofmodernmaritimesystems.
http://www.transnav.eu
the International Journal
on Marine Navigation
and Safety of Sea Transportation
Volume 17
Number 2
June 2023
DOI:10.12716/1001.17.02.02
270
responsetocyberincidentsthatareconsiderateofthe
organisational management processes. It is therefore
vital for management to provide procedures that
allow the crewto be ableto recognise,respond and
recovereffectivelyfromacyberincident,whetherthe
incidentisdeliberateoraccidental.
Developed through engagement with a
large
offshore operator and a national coastal
administration,thispaperproposesamaritimecyber
risk decisionmaking tool, the Cyber Emergency
ResponseProcedure(CERP).Basedonanoperational
flowchart,theCERPintendstoservethreepurposes.
Firstly, it provides a blueprint that allows
organisations to include cyber incident response
within
their standard incident response procedures.
Allowingthedevelopmentofpoliciesandprocedures
thatareconsiderateofprocessesandpracticesalready
in place.Secondly, it provides ahighlevel decision
makingtoolthatguides crewthroughtheresponseto
a cyber incident. This tool guides the crew through
the initial identification of
a cyber incident, and
managingitssymptomsandoutcomesusingstandard
documentation found on board. Thirdly, the CERP
sets outto demonstrate theneed for,and procedure
forattaining,externalsupportinthefaceofacyber
incidentthecrewcannothandleindependently.
The rest of the paperis as
follows.Section2will
explore the current approach to current maritime
incident response and cyber incident response,
justifyingtheuseofaflowchartliketheonepresented
in this paper. Section 3 will present the CERP and
demonstrate its implementation through the use of
examples.Section4willexplorethefuture
workthat
wouldberequiredtoeffectivelyimplementtheCERP
into maritime operations.Section 5 will conclude by
arguingthattheCERPisavitalfirststeponalonger
road to effective emergency response to maritime
cyberincidents.
2 MARITIMEANDCYBERINCIDENTRESPONSE
Theresponsetomaritimeincidents
isheavily driven
byregulatorybodiesandinternationalrequirements.
As such, this section will start by introducing the
current maritime incident response and some of the
tools, like checklists, that have been standardised in
anattempttoaidthatresponse.Thesectionwillalso
investigateseveralofthekeycybersecuritystandards
thatprovidesomeinsightintothedevelopmentofan
appropriate cyber incident response. Finally, the
sectionwillexplorehowthesectoriscurrentlycoping
with maritime cyberrisk andlay thefoundations of
howtheworkofthispapercanenhancethatresponse.
2.1 Maritimeincidentresponse
The current response
to a maritime risk event is
illustrated in Figure 1, whereby in the event of an
incidenttheprimaryobjectiveistoensurethesafety
of the vessel and crew through the use of incident
procedures.Ifcompletedcorrectlythisshouldleadto
the safe conclusion of the incident, whereby
operations
will continue as normal, or in a reduced
mode. For simplicity, this paper will adopt the
followingdefinitions.Returningtonormaloperations
meansthattheincidenthasnotlimitedtheoperation
ofthevesselandnofurtheractionwouldberequired.
Reduced mode covers all other outcomes including
theneed
togainoutsideassistanceinordertoreturn
tonormaloperations.
Ensure safety
of vessel
Risk event
Normal incident
procedure
Reduced mode
Normal operation
Figure1.Traditionalincidentmanagement
As theUN regulator chargedwith governing the
maritimesector,theIMOhasdevelopedavarietyof
regulatory frameworks to improve the safety and
security of the sector [8]. The framework most
relevant to this article is the International Safety
Management (ISM) Code [9], which is mandated
underChapterIXof
theInternationalConventionfor
the Safety of Life at Sea (SOLAS) [10]. The primary
aim of the ISM Code is to guarantee, preserve and
embedmaritimesafetyandpollutionpreventioninto
everyday maritime operations [11]. One particular
requirement of the ISM Code obligates companies,
and their vessels, to implement, and
maintain, a
Safety Management System (SMS). Failure to
implement an SMS will result in the vessel being
unable to obtain its Safety Management Certificate
(SMC) and subsequent Document of Compliance
(DoC),hinderingitsabilitytooperate.
AcompliantSMSprovidescrewwithmeasuresto
respond at any time to accidents, hazards,
and
emergency situations, such as fire, grounding, and
collision. Through the useof risk assessments, these
measures are adapted by each company to be
considerate of operational constraints and
organisational structure. As part of this process,
companies should identify response procedures to
emergency situations, and establish drills and
exercises to
practice them [9]. For the offshore
operatortheauthorsengagedwith,thesedrillswere
on a trimonthly basis and were complementary to
othersafetydrills,likefireorevacuation.
Partoftheresponseproceduresandplansinclude
the use of checklists that detail the process through
which the expected,and essential,
actions shouldbe
taken to manage the incident [12]. For example, see
Figure 2 which details the contents of the checklist
action plan that is to be used in response to a
suspectedransomwareattack.
271
Figure2.Examplemaritimechecklist
As the example illustrates, each checklist is
designed for a specific incident, in this case, that is
ransomware,butothersincludesensorfailuresorfire.
Thechecklistprovidesabriefdescriptionoftheriskto
outline the parameters that this checklist is
appropriate for. The final, and most important
elementis
theactionplan,whichprovidesclearsteps
thatthecrewshouldtakeinresponsetotheincident.
These actions should be developed in collaboration
with both crew and onshore management to ensure
theresponseisbothappropriatetotheoperationsand
considerateoftheexistingorganisationalpoliciesand
procedures.
2.2
Cyberincidentresponse
Cyber security and information security have gone
handinhandformanyyears.Tothisend,therearea
number of key documents, both regulations and
standards, that have been published to provide
insight into improving the cyber security of digital
systems.TheISO27000series,consistingof
multiple
standards, are one of the most iconic within the
domain. The introductory ISO 27000 provides the
highlevel terms of reference for the security
management of any system that collects, processes,
stores and transmits information [13]. ISO 27001
provides the requirements for establishing,
implementing, maintaining, and improving such
information
security management systems. These
requirements include the establishment and practice
of procedures that allow for a quick, effective, and
orderly response to information security incidents
[14].Insection5.24,ISO27002providesmoredetails
on the development of incident plans. The standard
arguesthatorganisations shouldestablish plansthat
are considerate
of the organisationʹs specific risks,
capability for detection and response, as well as
ensuring appropriate training is identified, and
deliveredtothoseexpectedtorespond[15].
Arguably the ISO 27000 series focuses on
Information Technology (IT) systems and not the
Operational Technology (OT) systems commonly
found onboard ships. However,
many of these OT
systems areunderpinned by IT systems and require
accurateandreliabledata(i.e.,information)tooperate
effectively. Therefore, highlevel security
requirements, like response plans, are easily
transferable between the IT and OT space. Whilst
standards are useful for providing guidance for the
developmentof incidentresponse
practices, theyare
onlyvoluntaryrequirements.
In 2016, the EuropeanCommissionpublishedthe
Network and Information Security (NIS) Directive
(EU Directive 2016/1148), which lays down
requirements that certain organisations within the
EuropeanUnionmustadheretoinordertoraisethe
levelofsecurityofnetworkandinformationsystems
[16].
At the start of 2023, the EU Commission
published NIS2 which will replace the original NIS
Directivewhenitentersintoforcein2024[17].Within
NIS2, there are clear requirements for organisations
defined as either “essential” or “important” to have
cyberincidentresponseplans.Theseplansthemselves
must include reporting
mechanisms of incidents to
the national authorities. Again, highlighting how
cyber response procedures do not only require the
involvement of the operator but often include the
involvementofexternalstakeholders.
The above documents, whilst reiterating the
importance of having cyber incident response plans
do not provide clear details on what
these plans
should include aside from the potential need to
report. The National Institute for Standards and
Technology (NIST) Cybersecurity Framework [18],
whilst again having an IT focus, does provide some
details on what these plans should contain with the
ʺRespondʺfunction.Severalactivitiesareparticularly
relevanttothecontext
ofthispaper.Firstly,personnel
shouldknowtheirroleandtheorderofoperationsin
responsetoanincident.Therefore,theavailabilityof
checklistsdetailingproceduresisausefultool.There
should also be coordination between stakeholders,
both internallyand externally,toensure aneffective
response.
2.3 Maritimecyberincidentresponse
The maritime industry has for a long time been
vulnerable to cyber security risks, and over the last
few years regulations and requirements have been
implemented to reduce these risks. Whilst this
resolutionmarkstheformalneedfororganisationsto
considercyberrisk,arguablyothershadbeenpushing
this approach
for manyyears prior. Forexample, in
2011 the European Union Agency for Cybersecurity
(ENISA) published one of the earliest reports
highlightingthe sectorʹscyber security risks andthe
need for plans to be developed [19]. In 2016, the
maritimecybersecuritydiscussionintensifiedwitha
plethora of documents calling
for more action were
published. Firstly, classification society DNV
published their Recommended PracticeʺCyber
securityresiliencemanagementforshipsandmobile
offshore units in operationʺ [20]. Secondly, IACS
publishedʺIACS166 Recommendation on Cyber
Resilienceʺ [21]. Thirdly, BIMCO published the first
versionoftheʺGuidelinesonCyberSecurityOnboard
Shipsʺ
[22]. Such were the popularity of these
documentstheyhaveallsincebeenupdated,withthe
BIMCOguidelinesnowontheirfourthedition[5].
Following increasing pressure for action from its
membership,theIMOpublishedʺMSCFAL.1/Circ.3
272
Guidelineson maritime cyber risk managementʺ [3],
which provides highlevel recommendations on
maritimecyberriskmanagement.Thefollowingyear,
afterintensediscussiontheIMOratifiedMSC.428(98),
makingcyberriskmanagementamandatoryelement
withinaship’sSMS[4].Thisrequirementmeantthat
from 1st January 2021 in order to
obtain their DoC,
shipowners were required to consider their cyber
risks within their SMS and subsequently develop
plansandprocedurestomanagethoserisks.
Both these IMO documents argue that the sector
should consider “industry best practice” when
addressing cyber risk. Thus, the IMO recommends
operators consider the NIST Cybersecurity
Framework, the ISO 27000 series and the BIMCO
guidelinesasawaytoinformtheirpractices.Inlight
oftheentryintoforceofResolutionMSC428(98),the
ISO has released ISO 23806:2022, which focuses on
cyber safety for ships and marine technology [23].
Again,liketheotherdocuments,therearefew
details
inthespecificsofcyberincidentresponse.However,
the standard does present a highlevel cyber safety
riskassessmentthatallowsthecompanytodetermine
the specific risks that they face andmitigate against
those.
Some states, like the USA, have produced
documentation outlining their expectations for ships
that
are compliant with Resolution MSC.428(98).
Produced by the US Coast Guard (USCG), a Work
Instruction (WI) entitledʺVessel Cyber Risk
Managementʺ(CVCWI027)stipulatestheexpectation
thatallcompaniesshouldmaintainaVesselSecurity
PlanalongsidetheSMS,bothofwhichshouldinclude
cyberrisk[24].Theseplansshouldincludea
training
elementtoensurecrewareabletorespondeffectively
toacyberincident.TheWIalsoprovidessomedetails
onwhatthatresponseshouldlooklike,includingthe
need to request assistance from Coast Guard Cyber
ProtectionTeamandPortStateControlOfficerwhen
appropriate.
The previously listed documents
focus on
developing cyber incident response plans for ships
thatarecurrentlyoperating.AsmentionedinSection
1, IACS has been proactively developing new cyber
risk management requirements for new builds post
2024.BothURE26(cyberresilienceofships)andUR
E27 (cyber resilience of ships equipment) stipulate
that all
new builds classified by an IACS member
shouldhaveanincidentresponseplan[25,26].These
plans shouldʺ…contain documentation of
predetermined set of instructions to detect, respond
to, and limit consequences of incidents…ʺ [25, page
18].AsperURE27,theseplansshouldbedeveloped
considering the vesselʹs
operational requirements as
well as key information available from the
manufacturer.
Therefore,whilstmaritimecyberincidentresponse
forms part of the mandated requirements for ships,
there is still little information available as to what
these plans should include. What is clear, is that
failure to comply with the development of cyber
response instructions, and drills to test them, could
leadtononcompliancewhichwouldhaveanegative
impact on the operation of the vessel. To ensure
compatibilitywith currentpractices these new plans
should resemble the existing documentation for
incidentresponse.Thus,theseplansandinstructions
should take the form
of checklists and flowcharts
which support the decisionmaking process of crew
duringincidents.
3 ACYBERINCIDENTDECISIONSUPPORTTOOL
The previous sections have discussed there is little
work currently beingdone in applying the response
tocyberincidentstomaritime operations.Therefore,
thecoreaimofthispaperis
tointroduceamaritime
cyber incident response framework that can aid
organisations in the development of their own
response plans thatare considerate ofthe company
specific nuances of their operations, systems, and
crews.
In keeping with the traditional methods as these
represent both best practice, and the most effective
methods of responding to maritime incidents, the
authors considered the development of a checklist
thatwouldprovidedetailsonthehandlingofacyber
incident. However, following discussions with a
variety of stakeholders, including a large offshore
operator and coastal administration, it was decided
thatin isolation these checklists wouldbe
of limited
benefit. What was clear from these discussions was
that crews and organisations, while capable of
creating and completing checklists, do not fully
understand the correct procedure for dealing with
cyberincidentsatlarge.Thus,theauthorsdecidedto
developacyberriskdecisionsupporttoolthatfulfils
the
threepurposeslistedinSection1:
1. Actasablueprintfororganisationstoincludecyber
incident response within their existing response
procedures;
2. Provide highlevel decision support to crews
respondingtoacyberincident;
3. Demonstrate the role that external support will
playwithincyberincidentresponse.
The decided
format for this support tool,
mimickingthenormwithinthesector,isaflowchart
identified as the CERP (Cyber Emergency Response
Procedure). Asargued by [27], flowchartsprovide a
visualrepresentationoftheproceduresallowingcrew
toaddressrisksrationallyandsystematically.
3.1 CyberEmergencyResponseProcedure(CERP)
flowchart
By
introducing the maritime cyber risk decision
support framework in this way, the authors
emphasizethatthehandlingofcyberriskshallnotbe
prioritized before safety critical incident processes.
Aligned with the requirements of Resolution
MSC.428(98)[4]cyberrisksshouldsimplybeincluded
in the existing incident handling procedures, as any
otherrisk,suchasfireorflooding.Thesafetyofthe
vessel,crew,andtheenvironmentare,asalways,the
priority.
RememberingFigure1thatpresentedasimplified
emergency response procedure on board. Figure 3
takes this one step further and illustrates how the
crew should initiate the CERP if
there is aʺcyberʺ
element to the incident. In some situations,
273
particularly timecritical incidents, it may not be
possibletoinitiatetheCERPimmediately.Therefore,
thecrewʹsfirststepshouldbetoensurethesafetyof
theship,crew,andenvironmentbeforeattemptingto
initiatetheCERP.Forexample,considerthefollowing
ransomwarescenario.
Ensure safety
of vessel
Suspect cyber
incident?
No
Success
Yes
Risk event
Failure
Normal incident
procedure
Cyber
Emergency
Response
Procedure
Reduced mode
Normal operation
Figure3.Traditionalincidentresponseexpansiontoinclude
cyberincidentresponse.
A vessel is currently underway and suddenly all
thebridgeequipmentscreensdisplayanimagesaying
all their systems are encrypted until a ransom has
beenpaid.Thecrewrealisethatthismeansthatthey
have now effectively lostcontrol of the steeringand
propulsionsystemsofthevessel.Thecrews’
response
to this scenario, whilst clearly a cyber incident, has
two different potential routes depending on the
currentoperationalenvironment.Ifthisscenariowere
to occur whilst the vessel was transiting open seas
then,aslongasthereisnoimmediaterisktothecrew,
ship or environment, the crew
could initiate the
CERP.However,inthesamescenariobutthevesselis
now transiting a busy Traffic Separation Scheme
(TSS), the crew would need to ensure the safety of
theirshipandcrewaswellasothersbeforeinitiating
theCERP.Inthiscase,itwouldbetomanuallytake
control of the vessel and remove themselves from
danger, and eventually alert vessels in the vicinity
following their standard incident procedures. For
example, by the use of lights, horn, Automated
Identification System, Global Maritime Distress and
Safety System (GMDSS) and a PANPAN broadcast
viaVHF(i.e.,initiatingPANPAN
procedurebyvoice
via VHF). Once the shipand crew are safe then the
CERPcanbeinitiated.
The flowchart itself is developedconsideringISO
58071985[28],whichprovidesstandardisedsymbols
and definitions for flowcharts. Whilst the standard
does not fit the authorʹs purpose directly, the paper
hasadopted
theapproachunderthedescriptionofa
ʺProgram Flowchartʺ, whereby it details the
proceduralsequenceofoperationswithinaprogram.
Whilst this type of flowchart is best suited for a
computer program, in a simplified format it can
appropriately be used to visualise the procedure a
humanoperator can
followwithintheirownsystem
ofworking.
Figure 4 illustrates the CERP developed by the
authorsandverifiedwithexpertswithinthemaritime
sector. The CERP has 4 distinct phases, which also
relatetospecificdivisionson boardandashore.The
firstlabelledOperationalTeamistheinitialphaseof
the CERP. The operational crew, bridge, or engine
room have already determined that there is a
potentialcyberincidentoccurringandthatthesafety
ofthevesseliscurrentlynotatrisk.Withinthisinitial
phase, crew would be expected to identify the risk
(M1), this might be as simple as
identifying the
potentialsystem(s)atfault,orpotentialcausesforthe
consequencespresentedwithintheincident.Oncethe
system(s) at risk have been identified then the crew
needtodeterminewhethertheycanmitigatetherisks,
byeitherusingamanual/alternativemeasure(M2)or
isolatingthesystem(M3).Itis
notessentialthatboth
areachieved,butitcouldhelpreducetheriskofthe
incident spreading to other systems. Companies
wouldneedtoprovideproceduresforhowtoachieve
manual operation and isolation of systems, with
acceptablealternativeslisted.
The second phase labelled as the Onboard
Technical Response, is the
onboard crewʹs initial
attemptstomanage andmitigate thecyber incident.
Once the crew have identified the systems at fault,
they should be following prepared checklists and
procedures in troubleshooting the affected devices
(Doc1).Insomecases,thiswillwork,andtheshipcan
return to normal operations (T2).
However, if the
crewconsiderthereisapossibilitythattheproblemis
propagatedtoother systems, they should restartthe
CERPforthatparticularsystem.Thisshouldcontinue
untilcrewhaveexhaustedallpossiblesolutions.
Once this exhaustion has occurred onboard, the
crewshoulddeterminethatcontacting the Shoreside
Support
Teamfortechnicalsupportisthenextoption
(D4). These teams willcontain agreater expertisein
cyber incident handling or have access to this
expertise (contact with manufacturer support). In
somecases,thisshoresideteammaybeabletosolve
the incident remotely (T3), or by providing
instructionsto
thecrew,whowilleithersucceed(T2)
orfail.Onfailure,itmaybedeterminedthattheonly
possible solution would be to initiate thecompanyʹs
repair and replacement procedures (P2). In these
situations, the Master must consider the integrity of
theDoC.Forexample,iftheshiponlynavigates
using
an Electronic ChartDisplay andInformation System
(ECDIS) and does not have updated paper charts,
then the vessel could be deemed unseaworthy and
must, in the worst case, seek emergency harbour to
rectifydeficienciesintheDoC.
There are two important points of note that the
crewshould
be aware of duringthe implementation
oftheCERP.Firstly,ifthesituationofeithertheshipʹs
operationalenvironmentorincidentchanges,thenthe
crew should reassess the safety of the ship and
determinewhetherpreventativemeasuresneedtobe
takenimmediatelybeforeproceedingwiththeCERP.
Secondly,thethree
terminationpoints(T2,T3andT4)
arelabelledasreducedmode/normaloperations.This
is because there will be situations whereby the risk
hasbeenmitigatedenoughtoanacceptablelevelthat
operationscancontinue,justatareducedlevel.
274
Figure4.FlowchartfortheCyberEmergencyResponseProcedure(CERP)
3.2 TheCERPinpractice
This section will present three scenarios that
demonstrate how the CERP can be utilised by
companies and crews to respond to cyber
emergencies. Thescenarios are written to begeneric
inorderforthereadertoadjusteachscenariototheir
own experiences and operations. For
instance, the
bridge scenario could target the MultiFunction
Displays (MFD) or the Dynamic Positioning (DP)
systems. Each scenario will illustrate the route
through the CERP that the crew will take (with
manual actions notated by M#) to reach each of the
terminationpoints(T2,T3andT4).
3.2.1 Compromised
nonessentialdevice
During normal operations, a computer suddenly
displays a ransomware message, and the crew
member is unable to access any files on the device.
ThecrewmemberimmediatelynotifiestheMasterof
theproblem.UsingtheCERP,theMasterdetermines
there isno direct impact onsafety and instructs
the
crewmemberto remove thenetwork(ethernet) cable
toisolatethedevice(M3).Asperthedocumentation
(Doc1), the Master notifies the engineer on board
responsible for IT systems of theproblem who then
takesresponsibilityfortroubleshootingandreporting
back to the Master. Having already isolated the
device,the
engineerrebootsthedevicefromabackup
and the computer is no longer infected (D1). The
275
Master confirms with the rest of the crew that no
other devices seem to be impacted, so assumes the
ransomware has not propagated (D2). Allowing the
vesseltocontinuenormaloperations(T2).
Figure5. Implementation of CERP (a: Section 3.2.1, b:
Section3.2.2,c:Section3.2.3)
3.2.2 FaultyGNSSsensor
During normal operations, the crew are actively
using the ECDIS for navigation and determines that
the observed position is not corresponding to the
otherpositionfixingmethods(i.e.,visualandradar).
The officer of the watch notifies the Master of his
concerns. The Masterdeterminesthatwhilst
there is
no risk to the safety of the ship, ECDIS is a critical
system so corrective action is required. As it is not
possible to isolate the ECDIS, the Master instructs
crew to useother positionfixing methods and posts
anextralookoutasanalternativetothedevice
whilst
itisbeingtroubleshooted(M2).Thecrewthenfollow
thetroubleshootingchecklistsforECDIS(Doc1).After
severalunsuccessfulattempts,thecrewcannotsolve
the problem (D3) and determined another device
might be at fault (D5). Crew determine that it is a
Global Navigation Satellite System (GNSS) sensor
causing the issue
(M4), so begin the CERP for that
device. After unsuccessful attempts to troubleshoot
theGNSSsensor(Doc1),theMasterinstructsthecrew
tousethebackupsensorandwithsupportfromshore
initiates the decommission and replacement
proceduresforthefaultyGNSSsensor(P2),allowing
theshiptocontinueoperations
atinareducedmode
(T2).
3.2.3 Enginecontrolroom(ECR)systems
WhenenteringtheEngineControlRoom(ECR)the
Chief engineer notices an Ecigarette plugged into a
USBportofthecontrolpanel.Unsureifthedevicehas
transferred malware onto the control systems, the
ChiefEngineerimmediatelynotifies
theMasterofthe
situation.TheMasterdeterminesthatallsystemsare
fullyoperationalsodeemsitnotappropriatetotake
alternativemeasuresorisolateasystem(M2,M3).The
engineer considers the appropriate checklists (Doc1)
whichinvolvesthenotificationoftheshoresideteam
(D4). The shoreside team implements their
own
procedures for remotely accessing the ECR systems
and running their own security checks (P1). They
determine that the systems have not been
compromised, so instruct the vessel to continue
operationsasnormal(T3).
3.3 Rolesandresponsibilities
As per the requirements of a shipʹs SMS, all crew
should be
aware of their responsibilities when
responding to an incident [9]. Furthermore, as this
paper has argued the response to a cyberincident
mightrequiretheinvolvementofshoresidepersonnel.
Therefore, all personnel, both on board and ashore
needtobeawareoftheirresponsibilitiestoensurethe
most effective response
to an incident whilst
maintainingthehighestlevelofsafety.
3.3.1 Servicetechnicians
The management level onboard a ship, primarily
theMasterandChiefEngineer,holdthehighestlevel
of responsibility for responding to incidents. While
both must work seamlessly in response to a cyber
incident,bothhaveslightlydifferentroles
toplay.The
Master’s primary role is to ensure the continued
276
safetyofthevesselandits crew withanoperational
focus.ItistheMasterwhocompletesthementalrisk
assessmenttodetermineiftheshipisinasafeenough
positionand/orstatetoinitiatetheCERP,orifother
action is required prior to initiation. The Chief
Engineer,
on the other hand, whilst still having a
responsibility for ensuring safety, will primarily be
focused on providing technical support during an
incident and completing mental risk assessments
regardingthecriticalityofsystems.
Inbothinstancesthemanagementlevel on board
will primarily fulfil a coordination role, pulling on
their
substantive experiences and training to direct
other crew members in their response. They would
alsobetheones responsibleforcontactingshoreside
assistance,asrequired.Thesepersonnelwouldalsobe
expected to synthesise the information from all
sources across the ship and ashore and disseminate
that back to others in the
form of instructions or
information.
3.3.2 Technicalteamonboard
Thetechnicalteamwouldbethosepersonnelwho
have clearly defined areas of responsibility which
play a critical role in the safe operation of a vessel.
These personnel include navigation officers and
members ofthe enginedepartment.These personnel
hold
several critical roles in the response to cyber
incidents. Firstly, as they are the operators of the
technicalequipment(handson),theyarelikelytobe
thefirsttodetectaproblem.Thesecondresponsibility
theyhaveistoensuretheycommunicatethisproblem
to the management level, along with any
other
operational information that could influence the
response.Thethirdandfinalrolethatthesepersonnel
willfulfilistheimplementationoftheresponse.Take
the example in Section 3.2.2, the technical operator
wouldbeexpectedtoimplementthetroubleshooting
documentswheninstructedbythemanagementlevel
andreportback
onitssuccess.
3.3.3 Shoresideassistance
With the complexity of many maritime systems
and the plethora of attack vectors, it would be
surprisingifthecrewonboardthevesselwereableto
respond to all cyber incidents independently.
Therefore, shoreside assistance should be available
whenneeded.
3.3.3.1 Companysupportteam
Operatorsshouldrecognisethatwhilstcapableof
responding to many incidents, the crew are
operational experts, not technical experts. Whilst
many operators have a team, commonly termed “IT
Support”, they may lack the operational knowledge
andskillslikecommunication,requiredtorespondto
incidents on a moving vessel
[29]. Therefore,
operatorsshouldensureashoresideteamthathasthe
correctoperationalandtechnologicalknowledgeand
skills is able to provide support to the crew when
needed. This team will have their own set of
proceduresforrespondingtoacyberincident.These
procedures may include the remote access
and
maintenance of a system or the communication of
moredetailed,andtechnical,instructionsbacktothe
vesselforthecrewtoimplement.
3.3.3.2 Servicetechnicians
The second part of the shoreside assistance
includes service technicians, either from 3rd party
service providers employed by the operator to
maintain the vessel systems, or members of the
technicalsupportteamsfromtheoriginalequipment
manufacturers. Again, operators should recognise
thattheirtechnicalstaffmayrequiretheassistance
of
thosemoreintimatelyawareofthesystemstoenable
an effective response. Operators have the
responsibilitytoensurethat,wheninvolvingexternal
support,informationispassedtotheseteamssothat
they can providea responsewhich is considerate of
the current operational requirements of the vessel.
The external
technicians have a responsibility to
comprehend this information and utilise the
knowledgewithintheirownorganisationstofacilitate
aneffectiveresponsetoanincident.
3.3.3.3 Othershoresideassistance
Whilstoutsideofthescopeofthispaper,itisalso
important to highlight that there might be other
stakeholderswhowouldbeinvolvedintheresponse
to a cyber incident onboard. This could include
entitieslikethecoastguard,military(orequivalency),
orotheroperatorsinvolvedinthe
rescueandrecovery
ofthevessel.Alltheseentitieshavedifferentrolesto
play, and operators should be aware of which
situationswouldrequiretheirinvolvementandhave
proceduresinplacetoinitiatethatinvolvement.
4 IMPLEMENTATIONOFCERPINTOMARITIME
OPERATIONS
The previous section illustrated the CERP and
demonstrated
how the CERP can function in a
practical,shipboardenvironment,affectedbyacyber
incident. However, to include the CERP fully and
safelyintomaritimeoperations,severalaspectsmust
be accounted for. The CERP must be tested and
verified in order to prove the integrity of the
flowchart, as well as
supporting documentation and
discussion of Cyber Emergency Response Teams
(CERT)trainingmustbeconsidered.
4.1 TestingandverificationofCERP
Twoperspectivesneedtobeconsideredforthetesting
and verification of the CERP. Firstly, there is the
verification of the CERP itself. Secondly is the
verification of the organisationʹ
s implementation of
theCERP.
In terms of validating the overarching CERP
framework, the authors presented the framework to
experienced operators who provided feedback and
comments.Allofwhichhavebeenimplementedinto
the final design, ensuring it is accurate at an
operational level. To further validate and test the
framework
more work must bedone byputting the
277
CERPintopracticeeitherviaworkshopsorsimulation
exercises with experienced crews. The use of these
simulatedexerciseswilldeterminewhethertheCERP
is a useful decisionsupport tool for crew to
understandtheirresponse.However,throughtheuse
of the threescenariosin Section 3.2,the authors can
demonstrate
how the CERP works in application,
providing a soft verification of results. Once further
validationhasoccurreditwillallowtheCERPtofully
fulfilitscorepurposes.
ForanorganisationusingtheCERPasablueprint
for their own cyber incident response, it should be
tested at all levels
of maritime personnel (support,
operational and management). To ensure effective
preparation and response, both shoreside and
shipsidepersonnelshouldparticipateinjointtraining
drills allowing technical and operational knowledge
to be shared. These drills will also illustrate how
decisionmaking processes may differ across the
response team. Thus, informing the development
of
organisational policy. What is more, through these
drills and practices the implemented CERP can be
amended and adapted as required by the
organisation. Coupling these results with a detailed
cyber risk assessment methodology like the NIST
CybersecurityFrameworkwillalloworganisationsto
understandcrucial systems, assets, threats and other
possiblemitigationmeasures.
Consequently,theutilizationofthistoolwillguide
the user through the collection of key information
about the cyber incident, affected systems, and
operational status. The application will besimilar to
the NIST Cybersecurity Framework [18], which is
recommendedbytheIMO,asit provides companies
with a
methodology that allows them to identify
crucialsystemsandassets,assesssystemsthreats,and
provide needed mitigation procedures. This
informationcanthenbeusedtoinformthedecision
makingprocessofthecrewinresponsetoanincident,
to either restore the system enabling a return to
normaloperationas
soonaspossible,orasafeenough
temporarilyreducedmode.
4.2 Developmentofchecklists
AsseeninSection2.1itisimportantforoperatorsto
follow industry guidelines as well as comply with
regulatoryrequirementsaddressingcybersecurity[5].
Onesuchrequirementisthedevelopmentofresponse
plans.WhilsttheCERP
representsapartofthatplan,
thispaperhasalsoidentifiedchecklistsasanessential
cognitive aid that has many benefits to incident
response. In safetycriticalindustries, checklists have
beendescribedasaʹfourthcrewmemberʹ[30].Thus,
when designed correctly checklists help users recall
criticalsteps,reducethe
stressexperiencedduringan
incident,aswellasmaintaineffectiveteamwork[31].
The BIMCO Cyber Workbook provides several
examplesofchecklistswhichincludeguidanceonthe
initial response, notification, and investigation of
cyber incidents on board [32]. However, these are
generic and should be used for reference by
organisations as they
develop their own which are
considerateoftheiroperationspecificrisks,including
the different IT and OT systems. This also includes
engaging with other key stakeholders like system
operatorsormanufacturers.
It is also important to note that whilst checklists
areuseful, theydohave limitations suchas they set
out explicitly the expected actions the crew should
take. However, from discussions with industry, the
authorsnotedthatinresponsetorealworldincidents
crew often act independently. This deviation, whilst
not exactlydesirable, might in certain circumstances
bethemostappropriateresponse.
Therefore, to help ensure these checklists are
appropriate
theyshouldbeimplementedduringdrills
and practices. This has two benefits, like the CERP,
firstly it allows the organisation to determine if
changesarerequired,andsecondly,itallowscrewsto
become familiar with their contents [33]. What is
more, practicing these checklists allows the practice
itself to be
reflected upon. As philosopher John
Dewey argues,ʺWe do not learn from experience…
welearnfromreflectingonexperienceʺ[34].
4.3 Developmentofcyberresponseteams
The roles and responsibilities of people engaging in
cyber incident handling are of importance, as
emphasisedinSection3.3.Thepaperhasarguedthat
to
ensureeffectiveincidentresponsededicatedcyber
responseteamsbothonshoreandonboardshouldbe
developed.
On the shoreside, the maritime industry is
increasinglyusing Security Operation Centres (SOC)
[35] which can benefit from implementing non
maritimecybersecurityspecialists[36].Asmentioned
in the USCG WI, the USCG have already
implemented Cyber Protection Teams, which also
support the maritime sector, not just landbased
companies[24].BIMCOhasputtheNISTframework
into a maritime context and specified that a cyber
emergencyresponseteam(CERT)shouldbeavailable
to provide timely support to the Designated Person
Ashore(DPA)[5,page53].
InIACSURE26,acyber
emergency response team is not specifically
mentioned.However,thedocumentdoesrequirethat
companiesimplementproceduresformanagingcyber
security incidents, and designate personnel with the
appropriate training and experience to respond to
suchincidents[25].
Regarding ships, it is not unreasonable to argue
thatthelinesofcommunicationtoshoresidesupport
maybeunavailable/compromised.Furthermore,with
seafarers fulfilling the role of operator they are
expectedtobringordertoanunnormalsituation[37].
Therefore, the authors argue that there should be a
dedicated CERT on board similar to the dedicated
firefighter on board. This
crew member should be
provided with specific incident response training,
which goes beyond cyber awareness. However, as a
2022 study found, there is a limited amount of
formalized training considering cyber risk in the
industry[7].Thus,operatorsshoulddeveloptraining
that provides key knowledge and skills regarding
cyber response,
that is considerate of the
organisationʹsoperations.
278
4.4 Training
As argued throughout this paper, certain skills are
required to implement the CERP. As the CERP
(Figure4)illustratestherearefourteamsrequiredfor
effectiveresponse.Eachoftheseteamsfulfilsdifferent
roleswithinincidentresponsethereforeneeddifferent
skills in ordertohandle cyber emergency situations .
Thus, different training modules will need to be
developed. As per roles and responsibilities, at the
managementlevel,thegeneralresponsibilityrelieson
the Master’s and Chief Engineerʹs operational
experience and team management skills. Therefore,
training must provide a detailed understanding of
cyberrisks,andmitigationmeasurestoallow
themto
identifypotentialincidentsanddirecttheappropriate
resources in response. At an operational level, the
onboard technical response team will need specific
details regarding systems, their dependencies and
troubleshooting methods. For the shoreside teams,
this training should include the skills required to
remotelyimplementmeasuresorcommunicatethose
mitigations
to the crew in the language they
understand.
Asargueddrillsandpracticesformavitalrolein
verifying and testing procedures, they also offer the
opportunityforpersonneltogainfamiliarizationwith
theskillstheyneedtodealwithabnormalsituations.
Thus, these drills can provide a dual purpose
in
training, allowing personnel to not only implement
response plans but also develop experiences which
canhelpinformtheirdecisionsatalaterdate.
5 CONCLUSIONS
This paper has investigated traditional maritime
incidenthandling,traditionalcyberincidenthandling
and maritime cyber security handling. Many of the
approaches discussed argue for the
need for cyber
incident response plans but fail to provide clear
detailsofwhattheseshouldcontain.Inresponse,by
analysing incident handling and taking a pragmatic
approach in collaboration with maritime industry
actors, the authors purpose a maritime Cyber
EmergencyResponseProcedure.Ascrewonboarda
ship is
traditionally known to take a pragmatic
approachtoproblemsolving,theflowchartprovides
the crew with a visual representation of a cyber
problemsolving approach, than a textbased
approach.
This flowchart serves three purposes. Firstly, the
CERPactsasablueprintfororganisationstoinclude
cyberincidentresponsewithintheir
existingresponse
procedures.TheproposedCERPisalsoconsiderateof
thetraditionalincidentresponseandbuildsuponand
adaptsbest practices to includeelements releva nt to
cyber incidents. Secondly, the CERP in its current
formatprovidesahighleveldecisionsupporttoolfor
crews, providing enough details of what steps
they
should be taking to safely manage a cyber incident.
These steps, again considerate of normal incident
response procedures, include the involvement of
shoreside support and the requirement to consider
whethertheincidenthaspropagatedtoothersystems.
Thirdly,theCERP illustrates whereexternalsupport
from the shoreside might be needed
in order to
respond appropriately. This support can come from
the technical support teams, equipment
manufacturers,orasintheUSCGexample,thestate.
In conclusion, the maritime sector lacks a
standardisedapproachtocyberincidentresponse.By
adapting current best practices, the CERP is a vital
first step to
addressing this issue. However, it is
important to note that this is just the first step on a
longer road to the effective emergency response to
maritime cyber incidents. Further work will be
neededtounderstandtheCERPʹs implementationat
an organisational level, as well as the training
required to
fulfil the roles and responsibilities it
highlights. However, the CERP does represent a
visual tool that will hopefully start muchneeded
discussions regarding maritime cyber emergency
response.
ACKNOWLEDGEMENT
This paper is partly funded by the research efforts under
MarCyandCyberMAR.
Maritime Cyber Resilience (MarCy) has received funding
fromtheResearchCouncilofNorway,withprojectnumber
295077.CyberMARprojecthas receivedfundingfromthe
European Union’s Horizon 2020 research and innovation
programmeundergrantagreementNo.
833389.
Content reflects only the authors’ view, and neither the
ResearchCouncilofNorwaynortheEuropeanCommission,
noranyproject partnerisresponsibleforanyusethatmay
bemadeoftheinformationitcontains.
The authors also want to thank the people at the Cyber
SHIP lab, Solstad
Offshore ASA, and The Norwegian
Coastal Administration for their engagement with this
research.
REFERENCES
[1]NORMA Cyber,ʺNORMA Cyber Annual Threat
Assessment 2022,ʺ Norwegian Maritime Cyber
Resilience Centre, normacyber.no, 2022. [Online].
Available: https://www.normacyber.no/news/norma
annualthreatassessment2022
[2]K. Tam et al.,ʺCase Study of a CyberPhysical Attack
Affecting Port and Ship Operational Safety,ʺ 2021, doi:
https://doi.org/10.4236/jtts.2022.121001.
[3]InternationalMaritimeOrganization,MSCFAL.1/Circ.3.
Guidelines
on maritime cyber risk management, 2017.
[Online]. Available:
http://www.imo.org/en/OurWork/Security/Guide_to_M
aritime_Security/Pages/Cybersecurity.aspx.
[4]International Maritime Organization, Resolution
MSC.428(98)‐Maritime Cyber Risk Management in
Safety Management Systems, 2017. [Online]. Available:
http://www.imo.org/en/OurWork/Security/Guide_to_M
aritime_Security/Pages/Cybersecurity.aspx. Accessed
on:22.02.2023.
[5]TheGuidelinesonCyberSecurityonboardShipsVersion
4.0, BIMCO, 2020. [Online]. Available:
https://www.bimco.org/aboutusandour
members/publications/theguidelinesoncybersecurity
onboardships
[6]IACS.ʺIACSadoptsnewrequirementsoncybersafety.ʺ
IACS. https://iacs.org.uk/news/iacsadoptsnew
requirementsoncybersafety/ (accessed 20 February,
2023).
[7]E. Erstad, M. S. Lund, and R. Ostnes,ʺNavigating
Through Cyber Threats, A Maritime Navigator’s
279
Experience,ʺ 2022, doi:
https://doi.org/10.54941/ahfe1002205.
[8]InternationalMaritime Organization.ʺMaritime Safety.ʺ
IMO.
https://www.imo.org/en/OurWork/Safety/Pages/default.
aspx(accessed20February,2023).
[9]InternationalMaritimeOrganization,Internationalsafety
management code: with guidelines for its
implementation, 2018 edition.; Fifth edition. ed. (ISM
Code). London: International Maritime Organization,
2018.
[10]International Maritime Organization, SOLAS,
Consolidated Edition, 2020 (SOLAS).
London:
InternationalMaritimeOrganization,2020.
[11]InternationalMaritimeOrganization.ʺTheInternational
Safety Management (ISM) Code.ʺ IMO.
https://www.imo.org/en/ourwork/humanelement/pages/
ISMCode.aspx(accessed23February,2023).
[12]InternationalChamberofShipping,Bridge Procedures
Guide.Marisec,2022.
[13]ISO/IEC27000:2018InformationtechnologySecurity
techniquesInformationsecuritymanagementsystems
Overview and vocabulary, ISO, iso.org,
2020.
[Online]. Available:
https://www.iso.org/standard/73906.html
[14]ISO/IEC27001:2017Informationsecurity,cybersecurity
and privacy protection Information security
management systems Requirements, ISO, iso.org,
2017. [Online]. Available:
https://www.iso.org/standard/82875.html
[15]ISO/IEC27002:2022Informationsecurity,cybersecurity
andprivacyprotectionInformationsecuritycontrols,
ISO, iso.org, 2022. [Online]. Available:
https://www.iso.org/standard/75652.html
[16]Directive (EU) 2016/1148 European
Union Parliament,
Official Journal ofthe European Union, 2016. [Online].
Available: https://eurlex.europa.eu/legal
content/EN/TXT/PDF/?uri=CELEX:32016L1148&from=E
N
[17]DIRECTIVE (EU) 2022/2555, European Union
Parliament, Official Journal of the European Union,
2022.[Online].Available:https://eurlex.europa.eu/legal
content/EN/TXT/PDF/?uri=CELEX:32022L2555&qid=167
7163438395&from=en
[18]Framework for improving critical infrastructure
cybersecurity, N. I. o. S. a. T. NIST, 2018. [Online].
Available:
https://www.nist.gov/cyberframework/framework
[19]ENISA,ʺANALYSIS OF CYBER SECURITY ASPECTS
IN THE MARITIME SECTOR,ʺ
https://www.enisa.europa.eu/publications/cyber
securityaspectsinthemaritimesector1,2011.[Online].
Available:
https://www.enisa.europa.eu/publications/cyber
securityaspectsinthemaritimesector1
[20]Cyber security resilience management for ships and
mobile offshore units in operation, DNV,
standards.dnv.com, 2016. [Online]. Available:
https://standards.dnv.com/explorer/document/0ED73B3
209DA42CDA6392BC3946585C9/4
[21]Rec166‐RecommendationonCyberResilience,IACS,
2020. [Online]. Available:
http://www.iacs.org.uk/publications/recommendations/1
61180/
[22]The Guidelines on Cyber Security onboard Ships
Version 1.0, BIMCO, 2016. [Online]. Available:
https://www.bimco.org/aboutusandour
members/publications/theguidelinesoncybersecurity
onboardships
[23]ISO23806:2022ShipsandmarinetechnologyCyber
safety, ISO, iso.org,
2022. [Online]. Available:
https://www.iso.org/standard/77027.html
[24]Vessel Cyber Risk Management Work Instruction,
United States Coast Guard, https://www.dco.uscg.mil/,
2020. [Online]. Available:
https://www.dco.uscg.mil/OurOrganization/Assistant
CommandantforPreventionPolicyCG5P/Inspections
ComplianceCG5PC/CommercialVessel
Compliance/CVCmms/
[25]IACS UR E26 Cyber resilience of ships, IACS,
https://iacs.org.uk/, 2022. [Online]. Available:
https://iacs.org.uk/news/iacsadoptsnewrequirements
oncybersafety/
[26]IACS UR E27 Cyber resilience of ships equipment,
IACS, https://iacs.org.uk/, 2022. [Online]. Available:
https://iacs.org.uk/news/iacsadoptsnewrequirements
oncybersafety/
[27]T.r.Qin,W.j.Chen,andX.k.Zeng,ʺRiskmanagement
modelinganditsapplicationinmaritimesafety,ʺJournal
ofMarineScienceandApplication,vol.
7,no.4,pp.286
291,2008.
[28]ISO 5807:1985 Information processing
Documentation symbols and conventions for data,
program and system flowcharts, program network
charts and system resources charts, ISO, iso.org, 1985.
[Online]. Available:
https://www.iso.org/standard/11955.html
[29]M.Raimondi,G.Longo,A.Merlo,A.Armando,andE.
Russo,ʺTrainingthe
maritimesecurityoperationscentre
teams,ʺin2022IEEEInternationalConferenceonCyber
Security and Resilience (CSR), 2022: IEEE, pp. 388393,
doi:https://doi.org/10.1109/csr54599.2022.9850324.
[30]P.Greig,A.Maloney,andH.Higham,ʺEmergenciesin
general practice: could checklists support teams in
stressfulsituations?,ʺ(ineng),BrJGenPract,vol.70,no.
695,pp.304305,Jun2020,doi:10.3399/bjgp20X709373.
[31]D. L. Hepner et al.,ʺOperating room crisis checklists
and emergency manuals,ʺ Anesthesiology, vol. 127, no.
2,pp.384392,2017.
[32]BIMCO, International Chamber of Shipping, and
Witherby Publishing Group,Cyber Security Workbook
for On Board ShipUse‐4th Edition,
2023. Livingston:
WitherbyPublishingGroup,2023.
[33]F.S.Foundation.ʺFSFALARBriefingNote1.5,Normal
Checklists.ʺ SKYbrary Aviation Safety.
https://skybrary.aero/bookshelf/fsfalarbriefingnote15
normalchecklists(accessed21February,2023).
[34]G. Di Stefano, F. Gino, G. Pisano, and B. R. Staats,
ʺLearning by Thinking: How Reflection Can Spur
Progress Along the Learning Curve,ʺ Management
Science, Harvard Business School NOM Unit Working
Paper No. 14093, 2014, doi:
https://dx.doi.org/10.2139/ssrn.2414478.
[35]A. Nganga, M. Lützhöft, J. Scanlan, and S. Mallam,
ʺTimely Maritime Cyber Threat Resolution in a Multi
StakeholderEnvironment,ʺ2022.
[36]G. Stoker, J. Greer, U. Clark, and C.
Chiego,
ʺConsideringMaritimeCybersecurityataNonMaritime
Education and Training Institution,ʺ in Proceedings of
theEDSIGConferenceISSN,2022,vol.2473,p.4901.
[37]E.Erstad,R.Ostnes,andM.S.Lund,ʺAnOperational
ApproachtoMaritimeCyberResilience,ʺTransNav,the
International Journal onMarine Navigation andSafety
of
SeaTransportation,vol.15,no.1,pp.2734,2021,doi:
https://doi.org/10.12716/1001.15.01.01.