International Journal
on Marine Navigation
and Safety of Sea Transportation
Volume 2
Number 3
September 2008
327
Reform of Technical Subject Examination with
Multiple Choice – Essay Writing Exam Forms
Ngo Thien Hoa
Vietnam Maritime University, Vietnam
ABSTRACT: For ages, oral and essay writing exams have been used for evaluating the knowledge on
technical subjects given to the students at Vietnam Maritime University (VIMARU) and other universities in
Vietnam as well. These two examination forms showed a lot of weak points such as limited exam content, no
objectiveness and big consumption of time in evaluation. It is the time a reform needed to be done.
Reform of technical subject examination with multiple choice- essay writing exam forms is presented in this
research paper. Multiple choice exams have already widely been applied to the subjects in the fields as
economics, social sciences and foreign languages, etc, but rather humble to technical ones; because, using
only single, multiple choice exam forms for evaluating the students’ knowledge on such a subject is not quite
reliable or adequate. The situation will be improved with the examination reform. It was proved by carrying
out experiments at VIMARU from year 2003 up to now.
In this paper, the methods of preparation of multiple choice-essay writing combined exam questionnaires,
especially, multiple choice questionnaires of problems, and the organization of the combined exams are also
studied and presented.
1 REQUIREMENTS AND OBJECTIVES OF AN
EXAMINATION REFORM AT VIMARU
Usually, oral and essay writing exams are used for
evaluating the knowledge on learning subjects,
especially, technical ones which given to the
students at VIMARU as well as other universities
and colleges in Vietnam. With time, these two forms
of exams seem to be unsuitable.
Quite precisely, oral exams allow evaluating the
level of learning of each examinee about a whole
subject; nevertheless, they also have a lot of
limitations. First, due to examiners’ direct
interference, the objectiveness in giving marks is not
ensured. Second, it is not suitable to a big number of
examinees; because it is time-consuming and
tiresome. Therefore, the oral form has hardly been
applied to the midterm or final exams of all technical
subjects with the number of examinees large and
increasing every year like of VIMARU.
And quite reliably, traditional essay writing exam
forms can be used for evaluating the examinees’
knowledge on a learning subject, and now, are
widely applied to almost exams at VIMARU, but,
beside some advantages as time spent for an exam is
regulated and controlled and it is suitable for a huge
number of examinees, they showed rather many
disadvantages. One, the testing content in each exam
questionnaire set which consists of some essay
questions is rather narrow; it cannot cover the whole
subject; because of exam time limit. Two, marking
exam papers needs much time and work. Three,
owing to the evaluating process not free from
examiners’ subjectivity, exam results are not full
objective; (the figure of students’ petitions about
their exam results certified it). And four, it is
328
difficult to automate the process of marking exam
papers.
At VIMARU and other schools as well, multiple
choice exams have just been used for the subjects as
foreign languages, but others, especially, technical
ones; because, using only single multiple choice
exam forms for evaluating the students’ knowledge
on such a subject is not quite reliable or adequate.
Although multiple choice exams have a lot of strong
points as it is easy to fully automate the process of
marking exam papers; simple to ensure the
objectiveness of the exam results and possible to test
the whole content of a subject in a quite short exam
time, they also have some radical weak points as it is
impossible to get a really precise evaluation result;
because of probability in choosing right answer
options to exam questions and difficult to test the
examinees’ creativity and ability of self solving
posed problems or profundity of understanding and
applying the theory which they have learnt. In
addition, the preparation of exam questionnaires
requires a lot of work and time.
To improve the evaluation of examinees’
knowledge on learning technical subjects, the first
time, in VIMARU, a reform of examination has been
carrying out with a combination of multiple choice
and essay writing, of which the highlight is that the
multiple choice of problems were applied to final
exams. The reform aimed at making use of all
advantages of both the two exam forms; namely,
to increase the reliability of exam results through
intensifying testing contents at unchanged limited
exam time and improving the objectiveness in
evaluation, reduce learning by rate, and decrease
the time and work spent for marking exam papers.
2 EXAMINATION REFORM WITH
COMBINATION OF MULTIPLE CHOICE
(MC) AND ESSAY WRITING (EW)
The most difficult matters of the reform are how to
prepare the exam questionnaires for the multiple
choice component, especially, multiple choice
questions for the problem parts and to combine the
two components: multiple choice and essay writing,
so that they will be reliable and meet the
requirements of technical subject final exams. These
matters were hard studied, tried out, amended,
adjusted and finally, well solved through successive
exam experiments which are presented hereafter.
2.1 Exam experiments
Since year 2003, successive experiments have been
done on the final exams of a technical subject named
Mechanics of Machines which has learning credits
of 4.
The percentage of the MC - EW components
of the combination was changed from 30% MC -
70% EW, 40% MC - 60% EW to 50% MC - 50%
EW.
Also, the testing contents were different, with the
spotlight application of multiple choice of problems
of a technical subject to final exams, as below:
− 30% MC theory - 70% EW (30% theory + 40%
problem)
− 40% MC problem - 60% EW theory
− 50% MC theory - 50% EW problem
− 50% MC problem - 50% EW theory
And four data were surveyed:
% subject content tested in an exam questionnaire
set (%STC)
Exam result (ER): % very good (%VG), % good
(%G), % average (%A), and total % pass (%
P);
(%P = %
VG + % + %A)
Time spent for marking an exam paper (T/EP).
Figure of petitions about exam results in
percentage (% PR).
2.2 Experiment results
The data gained from the experiments in comparison
with those from a couple of years before are shown
in table 1 below:
Table 1
No
Academic
year
% MC
- % EW
in a
combnd exam.
%SCT
Exam. time
(min)
Number of
examinees
T/EP
(min)
% PR
01 2001-
2002
0%MC-
100 % EW
35 90 223 5.5 % VG
26.7 % G
40.3 % A
17- 20
6.17
02 2002-
2003
0% MC-
100% EW
35 90 328 5.9 % VG
25.4 % G
45.1 % A
17 - 20
7.0
03 2003-
2004
30% MC-
70% EW
55 90 434
3.1 % VG
30.3 % G
41.6 % A
13 - 15
4.25
04 2004-
2005
40% MC-
60% EW
70 90 511 2.7 % VG
9 - 11
3.05
329
40.1 % A
05 2005-
2006
50% MC-
50% EW
80 90 507
2.1 % VG
34.1 % G
38.7 % A
6 - 8
1.68
0
80
35
50
% MC
% SCT
Fig. 1. % SGT - % MC
0
6-8
17-20
50
% MC
% T/EP
(min)
Fig. 2. T/EP - % MC
0
1.68
7.09
50
% MC
% PR
Fig. 3. % PR - % MC
0
34.1
25.4
50
% MC
% G
5.9
2.1
% VG
40
74
% A
% P
Fig. 4. % ER - %MC
2.3 Assessment of experimental exams
The data collected from the successive experiments
on the final exams of Mechanics of Machines verify
the better of multiple choice-essay writing combined
exams. These forms of exams allow making use of
all the advantages and by the same time, reducing
the disadvantages of the two singles. Combined
exams are much less time-consuming and much less
tiresome; because of the remarkably shorter time
and less work spent for marking exam papers (with
the help of answer keys for the MC component,
mean time spent formarking an exam paper with
50% MC - 50% EW is about a third of the time
spent for that with 0% MC - 100% EW). Also, the
exam results are more objective; certified by the
figure of examinees’ petitions decreased sharply
(from 7.09% with 0%MC - 100% EW to 1.68% with
50% MC - 50% EW). Furthermore, the reliability of
the reformed exams is visibly improved; shown
through the much larger subject content examined in
an unchanged limited exam time (from 35% with
0%MC - 100% EW to 80% with 50% MC - 50%
EW), and exam results with % P vibrated lightly
(about values of 74 %). % G increased continuously
(from 26.7% G with 0% MC - 100% EW to 34.1%
with 50% MC - 50% EW). % VG decreased
noticeably with larger % MC (from 5.9% VG with
0% MC - 100% EW to 2.1% with 50% MC - 50%
EW); this can be explained by the difficulty and
testing subject content increased significantly with
the raise of %MC (therefore only few really
outstanding students who well master the whole
subject, can do excellently and have the very good
marks). Once more, it verifies the better reliability of
the combined exam forms and the reduction of
learning by rate.
Through the successive experiments, the exams
combined 50% MC & 50% EW give the most
optimal option. With the data of %SCT, %ER, T/EP
and %PR, the combination ensures the optimum for
330
the examination reform. More than 50% MC in a
combined exam is not introduced to the technical
subject; because the probability can make the exam
results less truthful and other standards and skills as
profundity of self - presentation of the knowledge,
self - ability and self - creativity in applying the
theory to solving the problems, etc, are insufficiently
tested.
3 QUESTIONNAIRE PREPARATION FOR MC –
EW COMBINED EXAMS
As mentioned before, the MC-EW combined exam
questions play a decisive role in assessing the
reliability of exam results. Therefore, they ought to
be thoroughly prepared.
3.1 Questionnaire preparation for essay writing
component
The questions for the EW component are essay
questions. They are unchanged in comparison with
those of the traditional single essay writing exams.
For them, the most significant is how to join the MC
questions to create well combined exam
questionnaire sets which satisfy the requirements
of an exam.
3.2 Questionnaire preparation for MC component
3.2.1 Testing content
3.2.1.1 Theory part
With own specific characteristics, the MC exam
questions for the theory part of a technical subject
are aimed at testing the examinees’ knowledge on
the following matters:
Precise understanding and good memory for
the concepts, definitions, theorems, principles,
classifications, characteristics, uses, applications
and so on.
Ability of logical analysis, synthesis and inference
for quick selection of the right answer option for a
formula, diagram, graph, design approach, and etc.
3.2.1.2 Problem part
Also, with own specific characteristics, the MC
exam questions for the problem part of a technical
subject are prepared for testing the examinees’ skills
on the following matters:
Accurate understanding and applying the theory
of the subject to solve a technical problem, but not
the skill of calculating precise results.
Ability of logical analysis, synthesis and inference
for quick selection of the right approach option for
solving a type of technical problems.
3.2.2 Types of MC exam questions
The types of MC exam questions used for
building up the questionnaires of the MC component
can be different as Most Accurate, True/False,
Extension and Odd One Out, and etc. The larger the
number of the answer options in each question is the
more reliable the exam results are. Nevertheless, the
experiments showed that the suitable number of the
options is not less than 3, and not more than 7 for
technical subject exams in a rather short limited
exam time; e.g., 90 minutes as set for the final exam
of Mechanics of Machines. Hereafter, some types of
MC exam questions applied to the subject are
presented.
3.2.2.1 The MC questions for the theory part
− True/False
Questions require selecting either the True
(Correct) or the False (Incorrect); e.g.:
a b c
Fig. 5.
1 T/F: At rotating surfaces, the friction (
F
) is
tangential to a circle of radius
ρ
termed the
friction circle. The point of tangency is such
that the moment of the force about the centre
opposes the relative rotation, Fig. 5a.
2 T/F: At rotating surfaces, the reaction (
R
) is
tangential to a circle of radius
ρ
termed the
friction circle. The point of tangency is such
that the moment of the force about the centre
opposes the relative rotation, Fig. 5b.
3 T/F: At rotating surfaces, the friction (
F
) is
tangential to the rotating surface circle of radius
r. The point of tangency is such that the
moment of the force about the centre opposes
the relative rotation, Fig. 5c.
− Odd One Out
Odd One Out questions require choosing which
answer option is the “odd one out”; e.g.:
Which of the following options is the “odd one
out “ answer for defining a machine:
331
1 A machine is a combination of resistant bodies
for doing work.
2 A machine is an arrangement of rigid parts for
doing work.
3 A machine is a combination of mechanisms for
doing work.
4 A machine is a combination of links for doing
work.
− Extension
Extension questions require selecting the right
option for completing a principle, definition,
theorem, formula, approach and etc; e.g.:
A kinematical chain is called a mechanism when:
1 the flame link is specified.
2 there is one fixed or stationary link.
3 there is one driving link.
4 the driven links are specified.
5 its motion is known.
6 its locus is specified.
− Most Accurate
Most accurate questions require choosing which
answer option is the best; e. g.:
Which of the following options is the best descrip-
tion of the inertia force (
I
P
), on a link AB having
mass
m
and moment of inertia I about G; the centre of
gravity, with linear acceleration
G
a
, and rotating with
an angular acceleration
ε
as shown in Fig. 6.
Fig. 6.
a.
I
P
is equal and opposite to
P
(
P
=
m
·
G
a
), as shown
dotted in Fig. 6a.
b.
I
P
is equal to
P
(
P
=
m
·
G
a
), as shown dotted in Fig.
6b.
c.
I
P
is equal and opposite to
P
(
P
=
m
·
G
a
), and at
distance h ( h = I.
ε
/
m
G
a
), as shown dotted in Fig. 6c.
d.
I
P
is equal and opposite to
P
(
P
=
m
·
G
a
), and at
distance h (h = I.
ε
/
m
·
G
a
), as shown dotted in Fig. 6d.
3.2.2.2 The MC questionnaire for the problem part
− Most Accurate
Which of the following options is the best
approach for finding the velocity of point C in a
mechanism with driving link’s position; φ
1
, constant
angular velocity; ω
1
, and all necessary dimensions as
shown in Fig. 7 ?
Fig. 7.
Velocity diagram to scale μ
v
c
P = b
a1
=
a2
a3
µ
v
c
v
pc =
µ
v
v
pa
A1
1
=
µ
v
A
v
pa
3
3
=
µ
v
A
v
aa
32
32
=
a
a3
a3
Fig.03b
Fig.03c
P = b
P = b
c
a
1
=
a2
a1
=
a2
c
Vector equations of
velocities
On velocity diagram to
scale μ
v
, Fig. 03a,b,c.
(1)
;.
21
OA
AA
vv
ω
==
v
A
1
r OA;
1
pa
=
pa
2
=
v
A
1
/
µ
v
(2)
v
A
3
=
v
A
2
+
v
A
32
;
v
A
32
// AB;
pa
3
=
pa
2
+
aa
32
;
µ
v
A
v
pa
3
3
=
;
(3)
v
A
3
=
v
AB
;
v
AB
r AB;
pc
=
v
C
/
µ
v
;
3
pa
=
v
A
3
/
µ
v
;
(4)
v
C
= -
v
A
3
.
AB
CB
;
v
C
r AB;
pc = ba
3 .
AB
CB
= pa3
.
AB
CB
a. Approach 1:
v
C
=
pc
.
µ
v
(shown in Fig. 7a)
b. Approach 2:
v
C
=
pc
.
µ
v
(shown in Fig. 7b)
c. Approach 3:
v
C
=
pc
.
µ
v
(shown in Fig. 7c)
d. None of the approaches a, b,c above is correct.
− Extension
The variation of the driving torque; T
d
and
resisting torque; T
r
of a machine shaft with a mean
speed; ω
m
is shown in Fig. 8. The moment of inertia
of the machine is negligible.
Select the best answer option for completing the
solution used for determining the moment of inertia;
I
f
of a flywheel attached to the machine shaft
Fig. 6.
a
b
c
d
Fig. 7a
Fig. 7b
Fig. 7c
332
to keep the coefficient of speed fluctuation of the
machine at the value of
δ
.
The moment of inertia of the flywheel; I
f
, is
determined with the following formula:
I
f
=
2
ω
δ
m
U
, where
U
is the excess energy
absorbed by the flywheel and determined as below:
a.
U
= C area F2 C = 40
2
π
×
= 20
π
J
b.
U
= C area F1 C = 40
4
π
×
= 10
π
J
c.
U
= C area F3 C = 40
4
π
×
= 10
π
J
d.
U
= C area F1 C + C area F2 C+ C area F3 C
T
(Nm)
T
d
T
r
0
Working cycle
π
π
2
ϕ
F
1
F
2
F
3
80
40
Fig.04
Fig. 8.
4 ORGANIZATION OF MC–EW COMBINED
EXAMS
4.1 Forms of combining the MC component
questions with EW component questions
in combined exam questionnaires
4.1.1 % MC and %EW
The percentage of combination can be different as
30% MC - 70% EW, 40% MC - 60% EW, and 50%
MC - 50% EW.
4.1.2 Content of combination
And also, the content of combination can be
various as below:
1 30% MC theory - 70% EW (20% theory + 50%
problem)
2 50% MC theory - 50% EW problem
3 30% MC problem - 70% EW (50% theory + 20%
problem)
4 40% MC problem - 60% EW (50% theory + 10%
problem)
5 50% MC problem - 50% EW theory
Notes:
In fact, it depends on the specific requirements of
each technical subject, the percentage of the theory
and problem can be regulated.
4.2 Performance of combined exams
To ensure the reliability of exam results, the MC -
EW exam questionnaires, especially, the MC ones
must be renewed almost every year. Although
the renewal of them on computer is not so hard,
but copies of new exam questionnaire sets cost a lot
(the financial condition of VIMARU cannot
afford it). To avoid doing it too often, the EW and
the MC questionnaire sets are made as two separate
parts. In this way, the combination of the two parts
can be changed variously. Besides, the MC
questionnaire sets are so prepared that they can be
reused for some more times. At an exam, the chosen
answer option to each MC question is written down
on the exam paper instead of marking right away
on the questionnaire set; e.g., 1-a, 2-d, 3-f, 4-b, etc
(1, 2, 3, 4, etc, - the question order numbers and a, d,
f, b, etc,- the chosen answer options respective to the
questions), and then, the same exam paper is used
for EW questions. After each exam, the whole MC -
EW exam questionnaire sets together with the exam
paper must be handed in.
5 FINAL CONCLUSION
1. After successive experiments on Mechanics of
Machines, the examination reform with MC-EW
combined exam forms was fruitful. The
experience received from the experiments is quite
reliable to apply not only to Mechanics of
Machines but also all other technical subjects.
2. It depends on the specific requirements of each
technical subject; the MC- EW combined exam
forms can be applied to midterm and final exams
with MC component from 30% up to 50%. The
option 0f 50% MC is suggested.
3. To ensure the reliability of the exam results, the
MC - EW combined exam questionnaires must be
well prepared and renewed quite often, especially,
the questionnaires of the MC component.
4. The reformed exam results are more objective
and more reliable than those of traditional single
oral, essay writing and multiple choice exams.
T
(Nm)
Working cycle
333
5. The MC exam questions for the part of subject
problems are quite truthful for evaluating the
examinees’ knowledge on a technical subject.
6. The processes of doing exams and marking exam
papers are less tiresome and less time-consuming.
7. The process of marking exam papers of MC
component can be partially or fully automated
with the help of keys or machines.
REFERENCE
Dinh Gia Tuong & Ta Khanh Lam, 1999, Theory of Machines
& Mechanisms, Ha Noi.
Bui Xuan Liem, 1991, Theory of Machines & Mechanisms, Ha
Noi.
John Harnnah & R.C Stephens: Theory of Machines &
Mechanisms
a. 1991, Elementary theory & Examples, London.
b. 1992, Advanced theory & Examples, London.
Joseph Edward Shigley & John Joseph Uiker, 2003, Theory of
Machines & Mechanisms, Second Edition.
The University of Guelph, 2006, The Learning Commons-
Multiple Choice Exams, Canada.
The University of York, 2006, Preparing for Multiple Choice
Exams, Canada.
The University of Melbourne, 2006, Understanding and doing
well in multiple choice exams, Australia.