605
1
INTRODUCTION
In the shipping industry, several serious accidents
includingthecapsizeofthe
Heraldof FreeEnterprise,
the
Exxon Valdes tragedy, the capsize of the Estonia
and others, have happened. The accidents have
shocked the public and attracted great attention to
ship safety. As serious concern is raised over the
safety of ships all over the world, the International
MaritimeOrganisation(IMO)hascontinuouslydealt
with safety problems in the context of operation,
management,survey,ship
registrationandtheroleof
administration.Theimprovementofsafetyatseahas
beenhighlystressed.Theinternationalsafety‐related
marine regulations and rules were produced to
prevent similar accidents from occurring (Wang,
2001). The
Heraldof Free Enterprise in 1987, for
example, greatly affected the rule developing
activities of the IMO (Cowley, 1995, Sekimizou,
1997). This finally resulted in the adoption of the
International Safety Management (ISM) Code. All
suchaccidents(i.e.
ScandinavianStarin1990,Estonia
in1994 etc.) highlightedthe roleof human error in
marinecasualitiesand,asaresult,thenewStandards
forTraining,CertificatesandWatckeepingforseafers
were subsequently introduced (Sekimizou, 1997,
Wang,2001).
InMarineEducation&Training(MET),theuseof
simulators (engine or ship’s bridge) is fact. Various
maritime educational standards (i.e. STCW, 95,
Manila 2011) allow the simulators using in
educational practice. The aim for the application of
simulatorsinMETisthetransportofcapacitywhich
is the possibility to adopt the dexterities that are
learnedinaframeoftrainingoneintheoperationof
a
vessel. Because no situation is always the same
with a previous experience, the fact that an
individual becomes more specialized with each
repetition of similar objective lies in the fact of
transport. Indeed a faith in theʺmake ofʺ transport
constitutesthebasicjustificationforallprogramsof
Experimental Research in Operation Management in
Engine Room by using Language Sentiment/Opinion
Analysis
D.Papachristos&N.Nikitakos
Dept.Shipping,TradeandTransport,UniversityofAegean,Greece
ABSTRACT:ThepaperarguesforthenecessityofacombinationMMRmethods(questionnaire
,
interview)and
sentiment/opinion techniques to personal satisfaction analysis at the maritime and training education and
proposes a generic, but practical research approach for this purpose. The proposed approach concerns the
personal satisfaction evaluation of Engine Room simulator systems and combines the speech recording
(sentiment/opinion analysis) for measuring emotional user responses
with usability testing (SUS tool). The
experimentalprocedurepresentedhereisaprimaryefforttoresearchtheemotionanalysis(satisfaction)ofthe
users‐students inEngine Room Simulators. Finally,theultimate goal of this research is to find and test the
criticalfactorsthatinfluencetheeducationalpracticeanduser’ssatisfaction
ofEngineRoomSimulatorSystems
andtheabilitytoconductfull‐timesystemcontrolbythemarinecrew.
http://www.transnav.eu
the International Journal
on Marine Navigation
and Safety of Sea Transportation
Volume 8
Number 4
December 2014
DOI:10.12716/1001.08.04.17
606
education. We assume that the dexterities and the
knowledge that is learned in a classroom can be
applied effectively in similar situations outside it
(Tsoumasetal.,2004).
METfollows certaineducation standards
(STCW’95/Manila 2011) for each specialty (Captain,
Engineer)andforeachlevel(Aʹ,Bʹ, C’).Its
scopeis
the acquisition of basic scientific knowledge,
dexterities on execution (navigation, route plotting,
administering the engine etc) as well as protecting
the ship and crew (safety issues and environment
protection issues). Specifically, the maritime
educationstandarddefinesthreecompetencylevels:
Management,functionandsupportwhileatthesame
time
it defines related dexterities. Every dexterity
level suggests the totality of the learning goals and
the goal definition is the basic characteristic of
training.Thesimplercompetencemakeupthemore
complexones.Thishierarchicalincreaseinthelevel
of dexterity places an austere framework for the
educator designer of lessons
in each marine school.
The introduction of simulators and other modern
training tools constitutes an important research
question on what degree it can fulfil all the
expectations set forth by the STCW’95 (IMO, 2003,
Papachristosetal.,2012a,Tsoukalasetal.,2008).
The paper argues for the necessity of a
mixed
approach to usabilityand educational evaluation at
theengineroomandproposesageneric,butpractical
frameworkforthispurpose.Inparticular,wepresent
a multi‐method approach for the usability and
educationalevaluationofmaritimeenginesimulators
thatcombinesthephysiologicaldatageneratedfrom
speech recording by using sentiment
analysis and
questionnaires and interviews. This combination of
thesemethodsaimsatthegenerationofmeasurable
results of complementary assessments of the user
experience(Papachristosetal.,2012a,Papachristoset
al.,2012b).
The proposed approach is generic, in the sense
that it can be the starting point for an integrated
usability& educationalevaluation ofthe interactive
technologies during in‐situ education, simulation
and pragmatic ship operation management. The
approachiscurrentlyappliedtoassesstheusability
& educational practice of ship engine room in
educationalscenarios.
2
THEORETICALBACKGROUND
ResearchinHCI(Human‐ComputerInteraction)has
created many methods for improving usability
duringthedesignprocessaswellasattheevaluation
ofinteractiveproducts(Koutsabasis,2011).Usability
hasbeendefinedbyISO9241as“theextenttowhich
aproductcanbeusedbyspecifiedusersto
achieve
specified goals with effectiveness, efficiency, and
satisfactioninaspecifiedcontextofuse”.Itiswidely
acknowledged that the efficiency and effectiveness
can be measured in an objective manner, i.e. in
specificcontextsofuseandwiththeparticipationof
representative user groups, and they are usually
definedin
termsofmetricslike:tasksuccess,time‐to‐
task,errors,learnability(inrepetitiveusetests),etc.;
whilethepersonalsatisfactionissubjectiveinnature
anddependsonthecharacteristicsoftheusergroups
addressed(TullisandAlbert,2008).
The simulators constitute acategory of
educational software and follow a methodology of
application in instructive practice. The user is
promptedbythesoftwaretoundertakeactiveaction
for the making of personal work in the computer
(simulator)or in practicing the system (adjustment)
(Crook, 1994, Solomonidou, 2001, Tsoumas et al.,
2004). Maritime Engine simulation allows the
creationofreal,dynamicsituationsthat
takeplaceon
ashipatseainacontrolledsurroundingwherenaval
machine officers can (Kluj, 2002; Tsoumas et al.,
2004):
1
practicenewtechniquesanddexterities
2
shapeopinionsfromteachersandcolleagues
3
transportthetheory ofa realsituations inasafe
operation
4
faceseveralproblemssimultaneouslyratherthan
successively, can learn by giving priority to
multipleobjectivesunderhighpressuresituations
andchangesituationsaccordingly.
Itisdifficulttoexpectthatasimulatorwillbein
place to achieve all the previously mentioned
expectations.Thehigherrequirementsofauserleads
to the
increasing complexity of engine room
simulators,highercostandtheirlongest‐lastingtime
ofgrowth.Ontheotherhandthefastchangesinthe
Engine room equipment and the control techniques
require a lot of flexibility in the architecture of
simulators. The
model Det Norske Veritas for the
Certification of Naval Systems of Simulators
proposes the following four classification of
simulators(Kluj,2002;Tsoumasetal.,2004):
Category A (ENG). Acomplete mission simulator
imitatesalltheprocessesoftheinstrumentsinthe
control room, with the use of functional stations
intheinstrumentroom.
CategoryB(ENG).Anobjectivesimulator,capable
of imitating various instrument processes in the
control room, but with the limited use of
functionalstationsinthecontrolroom.
Category C (ENG). A limited objective simulator
capableofimitatingcertaininstrumentprocesses
inthecontrolroomforthetrainingprocedure.
Category H (ENG). A special objective simulator
capable of imitating an operation and/or
maintenance of particular equipment of
instrumentsand/ordeterminedscriptsofapplied
mechanics.
The psychological research in the language
production, comprehension and development is
developedmainlyafter1960asaresultoflinguist’s
N.Chomsky(1957),researchongenerativegrammar.
Thepsycholinguistic
researchshowedthatlanguage
comprehension and production is not influenced
only from factors not related to their linguistic
complexity but also from the speaker’s/ listener’s
existingknowledgefortheworldaroundhim/her,as
well as by the information included in the extra
linguisticenvironment(PinkerandJackendorff,2005,
Vosniadou,2001).
Inpsychology,thetermemotiontendstobeused
for the characterization of rather short but intense
experiences, while moods and preferences refer to
lower intensity but greater duration experiences.
607
Modern scientific community suggests different
views concerning understanding emotional
mechanism like natural reactions, purely mind
process, or hybrid combination (Lazarous, 1982,
Zajonc,1984;Vosniadou,2001).Ingeneral,wecould
note that psychology considers the emotional
mechanism as a determinist mechanism that pre‐
requiresastimulus–causeincitedinthe
brainbyuse
of the neuraland endocrine system (hormonal), the
response–emotion(Malatesta,2009,Papachristoset
al.,2012b).
Investigating the emotional gravity of words
spokenbyaspeakeranddefineditsemotionalstate
(current or past) constitutes a state of the art issue.
Mostoftheemotionalstate
categorizationsuggested
concern the English language. To overcome this
problem,studieshavebeenconductedthatapproach
themattercross‐culturallyandstudytheassignment
of the categories to various languages. This
assignmenthasconceptualtrapssincethemannerin
which an emotional state is apprehensible; an
emotional state is influenced by
cultural factors as
well.Inaratherrecentcross‐culturalstudydoneby
Fontaine et al., (2007), 144 emotional experiences’
characteristics were examined, which were then
categorized according to the following emotional
“components”: (a) event assessment (arousal), (b)
psychophysiologicalchanges,(c)motorexpressions,
(d) action tendencies, (e) subjective feelings,
and (f)
emotionregulation.
International bibliography contains various
approaches – techniques (sorting algorithms)
concerning linguistic emotional analyses, which are
followed and are based mainly in the existence of
word lists or dictionaries with labels of emotional
gravityalongwithapplicationsinmarketing,cinema,
internet,politicaldiscourseetc(Lambovetal.,2011,
Fotopoulou et al., 2009). There are studies also
concerning sorting English verbs and French verbs
that state emotions based on conceptual and
structural‐syntactical characteristics.For the Greek
languagethereisastudyonverbsofGreekthatstate
emotions based on the theoretical framework
“Lexicon‐Grammar” that is quite
old and doesn’t
contain data from real language use; there are also
somestudiesconcerningGreekadjectivesandverbs
that state emotions and comparison with other
languages (French – Turkish) under the viewpoint:
Structural‐syntactical + conceptual characteristics.
More recent studies in Greek conducted
systematically the noun structures based on the
theoretical framework of “Lexicon‐Grammar” and
theestablishmentofconceptual&syntacticalcriteria
for the distinction and sorting of nouns based on
conceptual‐syntactical characteristics of the
structuresinwhichtheyappear(Papachristosetal.,
2012b).
Specifically, as sentiment analysis and opinion
mining applications tend to utilize more and more
the composition of sentences and to use the value
and properties of the words expressed by its
dependency trees, there is a need for specialized
lexiconswherethisinformationcanbefound.Forthe
analysisofmorecomplexopinionatedtextlikenews,
political documents, and (online) debates the
identification ofthe
attitudeholderandtopicare of
crucialimportance(ChoiandCardie,2008,Jiaetal.,
2009,MoilanenandPulman,2007).Applicationsthat
exploittherelationsbetweenthewordmeaningand
its arguments can better determine sentiment at
sentence‐level and trace emotions and opinions to
theirholders(MaksandVossen,
2012).
3
RELATEDWORKANDSCOPE
The effectiveness evaluation of the educational
software in the educational practice was mainly
based on the experience & analysis (pos‐itivistic)
methods, which accept that knowledge may be
attributed only to the objective reality existing re‐
gardlessofthevaluesandbeliefstheonesseekingto
discover
her. As it shown in the international
bibliography, the use of multiple methods of
evaluation is more effective and the combinatorial
use of quantitative and qualitative approaches
confines their weaknesses (Brannen, 1995, Bryman,
1995,Patton,1990,Retalisetal.,2005,Tsianosetal.,
2009).
Specifically,theMixedMethodsResearch(MMR)
employs a combination of qualitative and
quantitative methods. It has been used asa distinct
approach in the social and behavioral sciences for
more than three decades. MMR is still generating
discussions and debates about its definition, the
methodinvolved,andthestandardsforthequality.
Although still evolving, MMR has
become an
establish approach. It is already considered the 3
rd
research approach, along with the quantitative and
qualitative approaches, and has its own emerging
worldview,vocabulary,andtechniques(Fidel,2008).
Usabilitytestingproceduresusedinuser‐centered
interactiondesigntoevaluateaproductbytestingit
on users. This can be seen as an irreplaceable
usabilitypractice,sinceit
givesdirectinputonhow
realusersusethesystem.Usabilitytestingfocuseson
measuringahuman‐madeproductʹscapacitytomeet
itsintendedpurpose(Dixetal.,2004,Nielsen,1994).
Anumberofusabilitymethodshavebeendeveloped
andpromotedbydifferentresearchers(Neilsonand
Mark,1994).
Inliterature
wemeetasusabilityrequirementsor
goals:(i)Performance,(ii)Accuracy,(iii)Recall,(iv)
Stickiness and (v) Emotional response. Some
usabilitytestingcanbeaccomplishedthroughtheuse
of checklists, guidelines, and principles. Most
usabilitytestingmethods involvetestsonreal users
and require human observers to evaluate the
outcomes
ofthetest.Consequently,usabilitytesting
tends to be rather labor intensive (Norman, 2006).
Severalstudieshaveattemptedtocompareusability‐
testing methods in terms of their ability to identify
types of usability problems and their influence on
designers(JohnandMarks,1997).Questionnairesare
often regarded as an inexpensive and
convenient
way to gather data from a large number of
participants. As adopted by many HCI (Human‐
Computer Interaction) and usability engineering
practitioners, the attitude questionnaire can be
transformedintoasocalledsatisfactionquestionnaire
(Czaja and Blair, 1996, Ryu, 2005). On other hand,
Qualitativeresearchmethodsweredevelopedinthe
social sciences to enable researchers to study social
608
and cultural phenomena. Much qualitative research
is interview based. The interview used in usability
research for understanding of the goals, needs, an
activitiesofpeoplewhousetheproducts(Kantneret
al.,2003).Untilrecently,field usabilityresearchhas
not gained wide acceptance among usability
practitioners.
There is considerable work
on the ergonomic &
usabilityassessment of the human strain (Torner et
al., 1994) and the design and arrangement of ship
equipment. This work has few applications in
industry(Petersenetal.,2010)andnotyetresultedto
well established evaluation methods and cases.
According to Osterman et al. (2010)
“several models
and methods
have been developed to estimate costs and
benefitsof
ergonomics in other industries, butnostudies
were
found from the shipping industry”. More
specifically, these studies tend to report on usage
effects on health, safety and mental workload;
howevertheyofferlittleguidanceontheevaluation
methods and/or the design of the respective
technologyandequipment (devices) withrespectto
usability(Papachristosetal.,2012b).
In recent years many sentiment analysis and
opinionminingapplicationshavebeendevelopedto
analyze opinions, feelings and attitudes about
products, brands, and news, and the like. These
applications
mine opinions from different sources like online
forumsandnewssitesandfrommovie,productand
hotelreviews.Manyofthesetoolsrelyonmanually
built or automatically
derived polarity and
subjectivitylexiconsand,inparticularforEnglish,a
couple of smaller and larger lexicons are available
(MaksandVossen,2012).
Theselexiconsarelistsofwordssensesannotated
for negative or positive polarity like as
(Hatzivassiloglou and McKeown, 1997, Maks and
Vossen,2012):
(Subjective:negative)angry—feelingorshowing
anger;“angryattheweather;angrycustomers;an
angrysilence”,
(Subjective:positive) beautiful — esthetically
pleasing,
(Objective:no polarity) alarm clock, alarm — a
clockthatwakesthesleeperatapresettime,and
(Objective:negative) war,warfare—thewaging of
armed conflict against an enemy; “thousands of
peoplewerekilledinthewar”.
Many interesting works exist that focus on
extracting the opinions from the customer reviews.
Someworksfocusonperformingopinionminingto
identifythesemanticorientationofareviewoverall,
whereas others focus on identifying
and extracting
the opinionwords that willdetermine the semantic
orientation.Afewworksexistthatperformsentence‐
levelsentiment analysis(i.e.sentiment analysis that
is using words but is not extracting representative
features).Theopin ionwordsare classified
individually and then the polarity of the opinion
sentence is calculated
by combining the individual
opinionwordpolaritywhileinthesentimentofeach
sentence is analyzed by identifying the sentiment
expressions and subject terms. Sometimes the
opinions regarding the products may not be
explicitlymentionedonthecustomerreviewsitesbut
they exist in web blogs. Finally there exist a few
product‐rankingtechniquesbasedonopinionmining
of product reviews for specific languages, such as
Chinese(Eirinakietal.,2012,MeenaandPrabhakar,
2007,KimandHovy,2004,Zhangetal.,2007).
We propose a research approach for educational
and usability evaluation of marine simulators with
emphasis in personal satisfaction (usability view)
that combines speech recording for measuring
emotional user responses‐lexical analysis with
usability assessment. Certainly, the proposed
approach may require further adaptations to
accommodate evaluation of particular interactive
simulation systems. The
main elements of the
proposed approach include (Papachristos et al.,
2013):
1
Registrationandinterpretationofuseremotional
states
2
(Speechrecordingandlexicalanalysis(sentiment
processing)
3
(Usability/Satisfaction & Educational assessment
questionnaires
4
Wrap‐upinterviews(emotionalassessments).
ThePersonal(subjective)Satisfactionisadifficult
measuringfactor.Forthat,weuseamixedtechnique
byusingalanguagedimension(sentiment analysis)
with MMR methods (questionnaire & interview),
verifying measurementscan be accomplish inorder
toextractsaferconclusions(Papachristosetal.,2013).
4
APPROACH
Themainpurposeofthisapproachistheanalysisof
emotionalstateandtheinvestigationofthestandards
thatconnecttheuser’sSatisfaction‐Happinessbyuse
oral text (as the basis for the situation) in the basic
dipole:
happiness (satisfaction) – sad (non satisfaction).
Weusedtwomethodologicaltools:
Sentiment /opinion analysis (Natural Language
Processing)
MMR methodology (Qualitative‐quantitative
techniques)
Theapproachcontains:
Process
Researchtools
PersonalSatisfactionModeling,and
Sentiment/OpinionAnalysis
1
Process
The experimental process will include four (4)
phases(Fig.1):
Phase 1:Presentation of the acceptance
documentbytheuser‐trainee.
Phase2:QuestionnaireCompletion(user’sprofile
and assessment educational and technical
characteristics)bythetrainee.
Phase 3: a semi structured interview about the
emotionalstate
Phase4:SimulationScenario(run)
Phase 5: Completion the simulation (scenario)
through a semi structured interview & usability
questionnaire(SUSTool)
609
Figure1. The Experimental research process (as rich
picture)
2 ResearchTools
Weusedfour(4)tools:
User Profile Questionnaire (UPQ): it contains: a
personal profile (gender, age), personal
background (education, experience and work
experience)etc.
RecordingSpeechTool (RST):Useofamicrophone
for speech recording of spoken words (speech‐
text).
Evaluation Questionnaire: (EQ) it registration
view/viewpoint/attitude data by using a
questionnaire.
System Usability Scale Tool (SUST): The SUS is a
simple, ten‐item scale giving a global view of
subjective assessments of usability. It is often
assumedthataLikertscaleissimplyonebasedon
forced‐choice questions, where a statement is
made and the respondent then indicates the
degree of agreement or disagreement with
the
statementona5(or7)pointscale(Brooke,1996).
TheSUSscoreshavearangeof0to100(seeTab.
I)
Table1.SUSscale
_______________________________________________
No QuestionCoefficient
_______________________________________________
1 IthinkthatIwouldliketousethissystem 4
frequently
2 Ifoundthesystemunnecessarilycomplex 1
3 Ithoughtthesystemwaseasytouse1
4 IthinkthatIwouldneedthesupportofa 4
technicalpersontobeabletouse
thissystem
5 Ifoundthevariousfunctionsinthissystem 1
werewellintegrated
6 Ithoughttherewastoomuchinconsistency 2
inthissystem
7 Iwouldimaginethatmostpeoplewouldlearn 1
tousethissystemveryquickly
8 Ifoundthesystemvery
cumbersometouse 1
9 Ifeltveryconfidentusingthesystem4
10 IneededtolearnalotofthingsbeforeIcould 3
getgoingwiththissystem
_______________________________________________
3
PersonalSatisfactionmodeling
The personal satisfaction modeling contains 5
levels:
Very
dissatisfied
dissatisfied neutral
Somewhat
satisfied
Very
satisfied
1 2 3 4 5
Figure2.TheSatisfactionlevels
From these levels, we design the Research
Personal Satisfaction Framework (RPSF)
(Papachristosetal.,2013):
Figure3.TheRPSF
4 Sentiment/OpinionAnalysis
ThesemanticorientationbasedinaLexiconBase
(LB). This approach based a Greek Lexicon of
Emotions (“ANTILEXICON”) [22]. Suppose the
followparametersforsentiment/opinionprocessing:
Sw
IndW W / TotN

(1)
Sw
IndW W / TotN

(2)
Sop
W|W W

(3)
Sop
W|W W

(4)
where
W:numberwordswithsentimentoropinionloadper
text(positivepolarity+ornegativepolarity‐)
5
FIRSTRESULTS
This experimental procedure is a primary effort to
research the educational and usability evaluation
with emotion analysis (satisfaction) of the users‐
studentsinmaritimesimulators.
1
Object
The engine room simulator of Norcontrol
Kongsbergwas selected for the experiment. It
belong the Merchant Faculty of Maritime Academy
of Aspropyrgos for the education of merchant
engineers on management issues, operation and re‐
establishmentofdamagesinatypicalengineroomof
610
avessel.Itisconstitutedbythefollowingareas‐parts:
(i) instructional Room Workstation, (ii) Engine
Control Room Workstation, (iii) Exercise Room
Workstation and (iv) Sound System. The engine
room simulator allocates special software on
monitoring the operations of the system as well as
growth of instructive scripts (PPT 2000
‐ MC 90‐ III
simulator).Thematerial (hardware) thatis usedfor
the support of software is constituted of 6 HP
workstationsthatrunonfunctionalsystemHP‐UX.It
also allocates a number of terminals for the
concretisation of the adjustment (7 PCs). Moreover
printers are used for the printing
of operating
parameters of the simulation and the evaluation of
thestudents(Tsoumasetal.,2004).
2
Participants
The sampling was carried out between January
and February 2013. The samples consisted of 6
students‐users. They were subjected to a specific
experimentalprocedure(Dieselgeneratoroperation)
in engine room simulator and completed the
questionnaires and gave interviews (research
approach).
3
DataAnalysis
Thedata ofexperiment (EngineSimulator) come
fromthreesources:
questionnaires,
SUStool,and
Interviews(voicerecording).
Specifically,wehave
Table2.DataProfile
_______________________________________________
VariablesResults
_______________________________________________
Sex6men
Medicalprofile4people(glasses)
4people(myopia)
Educationalbackground 3people(generalHighSchool)
2people(TechnicalHighSchool)
1People(other)
YearofstudyFirstyear(5people)
Forthyear(1people)
Englishlanguage1peopleExcellentlevel
2peopleVerygoodlevel
3peopleGoodlevel
Computerknowledgelevel 5peoplebasiclevel
(MSoffice/Internetusing)
1peopleadvancedlevel
(computerprogramming,web
site
designetc.)
SimulationExperience 2people
_______________________________________________
The educational evaluation is show the next
Table:
Table3.EducationalEvaluation
_______________________________________________
very satisfied NeutralDissatisfied very
satisfieddissatisfied
_______________________________________________
infrastructure 0 31 11
Courses 0 23 10
Teaching 2 22 00
_______________________________________________
And the satisfaction about the experimental
procedureis:
Table4.SatisfactionEvaluation
_______________________________________________
very satisfied NeutralDissatisfied very
satisfieddissatisfied
_______________________________________________
Scenario 1 50 00
Simulator 4 20 00
_______________________________________________
4
Sentiment/OpinionAnalysis
The next tables display the measures of lexical
datafromsentimentprocessing:
Table4.LexicalAnalysisforScenarioSatisfaction
_______________________________________________
Verysatisfied Satisfied
_______________________________________________
TotalN(words)53236
UsingMdf(modifiers)YesYes
SumMdf[∑W
mdf]210
MeanMdf[∑W
mdf]22
TotalN/SumMdf26,523,6
∑W
s/op+ 831
∑W
s/op‐00
TotalN/∑W
s/op+6,627,61
TotalN/∑W
s/op‐00
_______________________________________________
Table5.LexicalAnalysisforSimulationSatisfaction
_______________________________________________
Verysatisfied Satisfied
_______________________________________________
TotalN(words)19990
UsingMdf(modifiers)YesYes
SumMdf[∑W
mdf]75
MeanMdf[∑W
mdf]1,752,5
TotalN/SumMdf28,418
∑W
s/op+ 2312
∑W
s/op‐ 00
TotalN/∑W
s/op+8,657,5
TotalN/∑W
s/op‐00
_______________________________________________
Table5.MDFwordAnalysis
_______________________________________________
Satisfaction
_______________________________________________
MostusedwordsScenario Simulator Total
FrequencyFrequencyFrequency
_______________________________________________
“πολύ(alot/very)” 347
“αρκετά(enough)” 437
_______________________________________________
The most used phrase in user’s answers in
sentimentanalysishasthisformat:
Phrase:(mdf|auxiliaryverb)+satisfied (5)
andopinionanalysis:
Phrase:mdf+adjective|noun|verb (6)
And finally, the Topology (P
TOP) for
sentiment/opinionphrases in user’s answers
extendingalltheanswerduetothesmallsizeofthe
answers(average:48,2words):
TotalN/peopleofsample=289/6=48,16 (7)
611
6
CONCLUSIONS
The main purpose of this research, is the
investigation of personal satisfaction of a user of
MET equipment (Engine room simulator) via the
assistance of language techniques but also other
methodslikeMMR(questionnaires‐interviews).
The main elements of the proposed approach
include: speech recording for sentiment/opinion
analysis, Usability testing
procedure (SUS),
Attitudes/views questionnaires. The first results are
shows:
The Total N / Sum Mdf Index depending from
personalsatisfaction(growingfromveryhigh
→
high satisfaction) in Scenario & Simulator
satisfaction.
The Topology (PTOP) for sentiment/opinion
phrases in user’s answers extending all the
answer.
The most used words in sentiment phrases is
“αρκετά(enough)”&πολύ(alot/very)(simulator&
Scenariosatisfactionformusersanswers)andthe
most used phrase in user’s answers has this
format:(mdf|auxiliaryverb)+satisfied(verb)
&mdf+adjective|noun|verb.
Very High personal satisfactionforsimulator
(majority) and high personal satisfactionfor
simulator(majority).
In sentiment/opinion analysis, we observe the
MeanMdfIndexis2approximatelyforallcases.
Theresearchcontinueswiththenumeralincrease
ofthesampleandthetotalprocessingandevaluation
oftheresearchfindings(qualitativeandquantitative
data). The proposed approach may require further
adaptationstoaccommodateevaluationofparticular
interactivesystem s.
REFERENCES
Brannen, J. 1995. Combining qualitative and quantitative
ap‐proaches: An overview, J. Brannen (ed.), Mixing
Methods: Qualitative and Quantitative Research.
UK:Avebury,3‐38.
Bryman, J. 1995. Quantitative and qualitative
research:furtherreflectionsontheirintegration,Mixing
Methods: Qualita‐tive and Quantitative Research.
UK:Avebury,57‐80.
Brooke,J.1996. SUS:A“quick
anddirty”usabilityscale.In:
Jordan, P. W., Thomas, B., Weerdmeester, B. A.,
McClelland (eds.) Usability Evaluation in Industry,
Taylor&Francis,London,UKpp.189‐194.
Choi, Y. and Cardie, C.2008. Learning with
compositionalsemanticsasstructuralinferenceforsub
sentential sentiment analysis, Proceedings of the
Conferenceon
EmpiricalMethodsinNaturalLanguage
Processing (EMNLP 2008), pp. 793–801, Honolul,
Hawaii.
Crook,C.1994.Computers andthe collaborative experience of
learning.London,Routledge.
Cowley,J.,1995.TheConceptoftheISMCode.Proceeding
ofManagementandOperationofShips:PracticalTechniques
for Today and Tomorrow, The Institute of Marine
Engineers,24‐25May1995,London,Paper1.
Czaja,R.andBlair, J. 1996. Designing surveys: Aguide to
decisions and procedures, Thousand Oaks, CA, Pine
ForgePress.
Dix, A. Finlay, J. Abowd, G. D. Beale, R. 2004. Human‐
ComputerInteraction,UK:PearsonEducationLimited.
Eirinaki, M. Pisa, S. Singh, J. 2012.
Feature‐based opinion
miningandranking,Journal of Computer and System
Sciences78,pp.1175–1184
Fidel,R.2008.Arewethereyet?:Mixedmethodsresearch
inlibraryandinformationscience,Library&Information
ScienceResearch,pp.265‐72.
Fontaine,J.R.Scherer,K.R.Roesch,E.B.Ellsworth,P.C.
2007. The world of emotions is not two‐dimensional ,
PsychologicalScience,18(2),pp.1050‐1057.
Fotopoulou,A.Mini,M.Pantazara,M.Moustaki,A.2009.
“La combinatoire lexicale des noms de sentiments en
grec moderne”, in Le lexique des emotions, I. Navacova
andA.Tutin,Eds.Grenoble :ELLUG.
HagertyM,
JustMA.1993.Constructingmentalmodelsof
machinesfromtextanddiagrams,JournalofMemoryand
Language32,pp.71‐42
Hatzivassiloglou,V.andMcKeown,K.B.1997.Predicting
the semantic orientation of adjectives, Proceedings of
ACL‐97,Madrid,Spain.
IMO‐International,MaritimeOrganization,2003.Issuesfor
trainingseafarersresultingfrom
theimplementationon
boardtechnology,STW34/INF.6.
ISO 9241,ʺErgonomics of Human System Interactionʺ,
http://www.iso.org.
John,B.E.,andMarks,S.J.1997.Trackingtheeffectiveness
of usability evaluation methods, Behaviour and
InformationTechnology,16,4,pp188‐202.
Jia, L. Yu, C.T. Meng, W. 2009. The effect of negation
on
sentiment analysis and retrieval effectiveness,
ProceedingsofCIKM2009,China.
Kantner, L., Sova, H.L., Rosenbaum, S. 2003. Alternative
Methods for Usability Research, SIGDOC’03, San
Francisco,California,USA.
Kim,S.‐M.andHovy,E. 2004.Determiningthesentiment
of opinions, in: Proceedings of the 20th International
ConferenceonComputationalLinguistics,COLING’04.
Kluj, S. 2002. Relationship between learning goals and
propersimulator,ICERRS5Paper.
Kotzabasis,P.2011. Human‐Computer Interaction:
Principles, methods and examples, Athens, Kleidarithmos
(inGreek).
Lambov, D. Pais, S. Dias, G. 2011. Merged Agreement
Algorithms for Domain Independent Sentiment
Analysis, Pacific Association, For Computational
Linguistics (PACLING 2011), Procedia‐Socila
and
BehaviouralSciences,27,pp.248‐257.
Lazarous, R. S. 1982. Thoughts on the Relation between
Emotion and Cognition, American Psychologist, 24, pp.
210‐222.
Malatesta,L.2009.“Human–ComputerInteractionbased
in analysis and synthesis optical data”, Phd Thesis,
Athens(inGreek),NTUA.
Meena, A. and
Prabhakar, T.V. 2007. Sentence level
sentiment analysis in the presence of conjuncts using
linguisticanalysis,in:Proceedingsofthe29thEuropean
ConferenceonIRResearch,ECIR’07.
Moilanen,K.andPulman,S.2007.Sentimentcomposition,
Proceedingsofthe Conference on Recent Advances in
NaturalLanguageProcessing(RANLP),Bulgaria.
Maks, I. and
Vossen, P. 2012. A lexicon model for deep
sentiment analysis and opinion mining applications,
DecisionSupportSystems53,pp.680‐88.
Nielsen,J.1994.UsabilityEngineering,AcademicPress Inc.
Nielsen, J, and Mack, R.L. (eds.) 1994. Usability Inspection
Methods,NewYork,JohnWiley.
Norman, K.l. 2006. Levels of Automation and
User
Participation in Usability Testing, Interacting with
computers,Elsevier.
Pinker, S. Jackendorff, R. 2005. The faculty of language:
what’sspecialaboutit?,Cognition,95, pp.201‐236.
Österman, C. Rose, L. Osvalder, A.L. 2010. Exploring
maritime ergonomics from a bottom line perspective,
WMUJournalofMaritimeAffairs,Volume9,Number
2,
pp153‐168.
612
Papachristos, D. Alafodimos, K. Nikitakos, N. 2012a.
Emotion Evaluation of SimulationSystems in
Educational Practice, Proceedings of the International
ConferenceonE‐LearningintheWorkplace(ICELW12),13‐
15June,NY:KaleidoscopeLearning,www.icelw.org.
Papachristos, D. Koutsabasis, P. Nikitakos, N. 2012b.
Usability Evaluation at the Ship’s Bridge: A Multi‐
Method Approach,
In Proceedings of 4
th
International
Symposium on “Ships Operation, Management and
Economics”‐SOME12, The Greek Section (SNAME), 8‐9
November2012,EygenideioFoundation,Athens.
Papachristos, D. Alafodimos, K. Kikilia, K. Nikitakos, N.
2013.Sentiment Analysis Use in The Satisfaction
Evaluation for Maritime Education, Proceedings of the
International Conference on E‐Learning in the Workplace
(ICELW13),
12‐14 June, NY: Kaleidoscope Learning,
www.icelw.org.
Patton, M. Q. 1990. Qualitative Evaluation and Research
Methods.CA:SagePublications.
Petersen, E.S. Dittman, K. Lützhöft, M. 2010. Making the
Phantom Real: A Case of Applied Maritime Human
Factors, Proceedings of SNAME SOME
2010,http://publications.lib.chalmers.se/cpl/record/I
ndex.xsql?pubid=133364 (last access 18November
2014).
Retalis, S.
(eds.), 2005. Educational Technology. The
advancedinternettechnologiesinlearningservice,Ath‐
ens:KastaniotisEditions(ingreek).
Ryu, Y.S. 2005.Development of Usabilities Questionnaires
for Electronic mobile Products and Decision Making
Methods, Phd Thesis, Blacksburg, Virginia, USA,
retrieving from
http://scholar.lib.vt.edu/theses/available/etd‐ 08212005‐
234205/unrestricted/ETD_Ryu_Final.pdf (last access 19
November2014).
Sekimizu,K.,1997.Currentwork
atIMOonformalsafety
assessment. Proceedings of Marine Risk Assessment: A
Better Way to Manage Your Business. The Institute of
MarineEngineers,London,8‐9April1997.
Solomonidou, X. 2001. Modern Educational Technology.
Saloniki,Kodikas(inGreek).
Tian,P. Liu,Y.Liu,M.Zhu,S.2009. Researchofproduct
ranking technology based on opinion mining,
Intelligent Computation Technology and
Automation,vol.4,InternationalConference,pp.239–
243..
Tsianos, N., Lekkas, Z., Germanakos, P., Mourlas, C.,
Samaras,G2009.AnExperimentalAssessmentofthe
Use of Cognitive and Affective Factors in Adaptive
EducationalHypermedia,IEEETransactionsonLearning
Technologies,Vol.2,
No.3,July‐September2009,pp.249‐
258.
Tsoukalas, V. Papachristos, D. Mattheu, E. Tsoumas, N.
2008. Marine Engineers’ Training: Educational
AssessmentofEngineRoomSimulators,WMU Journal
of MaritimeAffairs, Vol.7, No.2, pp.429‐448, ISSN 1651‐
436X,CurrentAwarenessBulletin,Vol.XX‐No.10,Dec.
2008,IMOMaritimeKnowledgeCentre,pp.7.
Tsoumas, N. Papachristos, D. Matheou, E. Tsoukalas, V.
2004. Pedagogical Evaluation of the Ship’s Engine
RoomSimulator,usedinapprenticemarineengineers’
Instruction,1
st
InternationalConferenceIT,Athens.
Tullis,T.andAlbert,B.2008.MeasuringtheUserExperience:
Collecting Analysing and Presenting Usability Metrics,
MorganKaufmann.
Torner, M. Almstrom, C. Karlsson, R. Kadefors, R. 1994.
Working on a moving surface—a biomechanical
analysisofmusculoskeletalloadduetoshipmotionsin
combinationwithwork,Ergonomic
s,Vol.37.
Vosniadou, St. 2001. Introduction in Psychology, Vol. I,
Athens,Gutenberg(inGreek),2001.
Wang, J. 2001. The current status and future aspects in
formalshipsafetyassessment,SafetySciences38,pp.19‐
30.
Zajonc, R. B. 1984. On the Primacy of Affect, American
Psychologist,39,pp.
117‐123.
Zhang, W. Yu, C. Meng, W. 2007. Opinion retrieval from
blogs,in:Proceedingsofthe16thACMConferenceon
Conference on Information and Knowledge
Management,CIKM’07,pp.831–840.