How to design objective tests

DESIGNING ENGLISH OBJECTIVE TESTS AT UPPER-SECONDARY SCHOOLS
OBJECTIVE TESTS
VS.
ESSAY TESTS
Essay Tests: Testees are asked to present their ideas in a long writing piece responding to the question assigned.

Objective Tests: Testees are asked to give short answers to a large number of questions with necessary information to answer the question provided.
TYPES OF
OBJECTIVE TESTS
Matching Items
E.g. Match the jobs with their definitions
a. farmer 1. This person looks after young children.
b. dentist 2. This person has pigs, cows and sheep.
c. babysitter 3. This person keeps our teeth white, clean, and healthy.
 Answer: a-2; b-3; c-1

TYPES OF
OBJECTIVE TESTS
2. Supply Items
E.g. The relative pronoun WHO is used to replace a _______.
 Answer: person
3. Short answer
E.g. What is the superlative form of GOOD?
 Answer: the best
TYPES OF
OBJECTIVE TESTS
4. True/ False Questions
E.g. Decide if the following statement is true (T) or false (F)
The present simple can be used to describe a habitual action.
 Answer: T

TYPES OF
OBJECTIVE TESTS
5. Multiple Choice Questions (MCQs)
E.g. Choose the best answer a, b, c, or d to fill in the blank.
The man _____ you met at the party is Mary’s boyfriend.

which
whose
whom
who
 Answer: c
ADVANTAGES OF OBJECTIVE TESTS
They are more objective than other procedures (less dependent on opinion).
They allow instructors to assess a large and representative sample of course material.
They measure most types of learning.
They allow reliable and efficient test scoring
ADVANTAGES OF MCQs
Can test knowledge quickly with large
groups
Can be used to give quick feedback on
student performance
Can be mechanically scored
Can be reused
With careful writing, can test higher
cognitive skills.
DISADVANTAGES
OF MCQs
Takes a lot of time to construct good
MCQs
Without care, MCQs tend to test rote
learning and recall only
Cannot test the ability to arrange,
justify and analyze ideas
Cannot test creativity
Tends to reduce the range of marks
MULTIPLE CHOICE QUESTIONS (MCQs)
Definition
Parts
Advantages
Disadvantages
Types
Uses
DEFINITION OF MCQs

“An MCQ question is a question in which students are asked to select one alternative from a given list of alternatives in response to a question stem”
MCQ TERMINOLOGY
Item: a question on a test
Lead-in: an instruction of an item
Stem: the question part of the item
Options: All the choices in an item
Key: The correct answer
Distracters: The incorrect options

PARTS OF AN MCQ
Choose the best answer a, b, c, or d to complete the sentences.
The man _________ is my brother.
whom you saved
who saved
that saved
whom saved
Lead-in
Stem
Key
Distracters
Options
PARTS OF AN MCQ EXCEPTIONS
Without a lead in
e.g. “Important” is synonymous with
a. trivial
b. significant
c. dramatic
d. considerable
Without a stem
e.g. Choose a word whose underlined part is pronounced differently.
a. century b. error
c. energy d. geology
QUALITIES OF
GOOD MCQs
Validity: Items measure only what they are supposed to measure.
Fairness: Items do not unfairly advantage or disadvantage certain groups or people
Sensitivity: Items avoid offensive or upsetting language, situations or actions
Difficulty: Items are written at appropriate levels of difficulty
TYPES OF MCQs
1. Select correct answer
2. Select incorrect answer
3. Fill in the blank
4. Match 1 or 2
5. Mismatched
6. Sequence of events

1. Select correct answer
Choose the correct passive transformation of the following sentence.
The others told the new students where to sit.
The new students were told where to sit.
The others were told where to sit by the students.
Where to sit was told to the new students by the others.
The new students were told where to sit by the others.
 Answer: d
2. Select incorrect answer
All of the following words can be followed by an infinitive EXCEPT for
Want
Remember
Enjoy
Hope
 Answer: c
3. Fill in the blank
Choose one word to fill in the blank.
Your birthplace is where you were ……..
living
born
visiting
working
 Answer: b
4. Match 1 or 2
Choose the correct pair of words to fill in the blank.
_____ student asked a question; _____ answer it.
a. One/ the
b. The/ one
c. One/ another
d. The/ another
 Answer: c

5. Mismatched
Which adjective is mismatched with its antonym?
consistent; unconsistent
like; unlike
sympathetic; unsympathetic
kind; unkind
 Answer: a
6. Sequence of events
Having finished the work, he turned off the computer and left the office after checking all the windows to make sure they were locked.
The right sequence of action is:
Finished the work; turned off the computer; left the office; checked the window
Finished the work; turned off the computer; checked the windows; left the office
Finished the work; left the office; turned off the computer; checked the window
Turned off the computer; checked the window; finished the work, left the office.
 Answer: b
COMMON ITEM PROBLEMS
Cueing
Impurity
Apples and oranges
Unparallel options
Gender representation
COMMON ITEM PROBLEMS
Stereotypes
Double keys
No key
Outside or Background Knowledge
Trickiness
Insensitivity
CUEING
Answer found within stem

When you write “for deposit only” on a check, you are probably going to …
cash a check
deposit money
transfer funds
open an accounts
IMPURITY
Items that test more than one thing

I never knew ________
had the boys gone
the boy have gone
have the boys gone
the boys had gone
DOUBLE KEY
More than 1 correct answer

The president _______ the election by a landslide.
won
he won
winning
could win
APPLES AND ORANGES
2 for 2 split

Nowadays people use mobile phones….
frequently
seldom
in their cars
for emergency purposes
UNPARALLEL OPTIONS
Length & Grammar

How do most Russians go to work?
By bus
They go by taxi
Most of them drive
Walk
GENDER REPRESENTATION
The mailman ________ many packages despite the rain.
to deliver
delivering
delivered
deliver
STEREOTYPE
People in the South are said to _______.
being mean
be mean
mean
be meant
NO KEY
There is a student _________ I told you about yesterday.
which
where
how
what


OUTSIDE BACKGROUND KNOWLEDGE
The weather in Siberia is ________ hot any sunny.
often
never
sometimes
rarely

TRICKINESS
Smoking has a bad _________ on health.
affect
effect
affliction
effectiveness
INSENSITIVITY
Many people _______ in the tsunami that struck Southeast Asia last month.
were killed
killed
are killed
to kill
USING MCQs TO TEST LANGUAGE SKILLS
Grammar
Vocabulary
Phonetics
Reading
MCQs TO TEST GRAMMAR
EXAMPLE
Testing point: The use of –ing participle
(Lesson 1/ Tieng Anh 12)

Choose a, b, c, or d to fill in the blank.
I don’t enjoy ____________.
listening to rock music
listen to rock music
to listen to rock music
my listen to rock music
 Answer: a
MCQs TO TEST VOCABULARY
EXAMPLE
Testing point: Adjectives with the same root
(Unit 2/ English 12)
Choose a, b, c, or d to complete the sentence.
She is very ________ with pain.
sensible
senseless
sensitive
sensational
 Answer: c
MCQs TO TEST PHONETICS
EXAMPLE
Testing point: and
(words taken from Unit 3/ English 12)
Choose one word whose underlined part is pronounced differently.
Sure
Stage
Basic
Sign
 Answer: a
MCQs TO TEST
READING SKILL
Testing point: skimming skill
(Adapted from lesson 1/ Tieng Anh 12)
Read the passage and decide what it talks about.
People use their free time in many different ways. Some enjoy going to social gatherings like parties or English clubs because they love being around with many people. Others want to have peaceful moments after a hardworking week. They prefer staying home reading a book or watch a movie. Those who love nature may like to go fishing or swimming in the river.
Different personalities
Different ways of spending free time
People’s likes and dislikes
Things to do when you have free time
 Answer: b
STEPS OF WRITING
A GOOD MCQ

Before you write
While you write
After you write
STEPS OF WRITING
A GOOD MCQ
BEFORE YOU WRITE
Decide on the testing points
Collect sufficient material to write test items.
Construct a design of the test: number of items, elements of language, skills and levels of difficulty.
STEPS OF WRITING
A GOOD MCQ
WHILE YOU WRITE
Writing the stem
Writing the lead-in
Checking the stem and lead-in
Writing the options
1. WRITING A STEM
1.1. The stem should test just one concept.
Weak stem: What are the comparative and the adverb form of the adjective “dramatic”?
 Strong stem: What are the adverb form of the adjective “dramatic”?
1. WRITING A STEM
1.2. The stem should include most of the wording to avoid repetition in the alternatives.
Weak stem: The present simple can be used to describe
an action that stops a short time before.
an action that is always true.
an action that refers to a future event.
an action that is finished.
 Strong stem: The present simple can be used to describe an action
That stops a short time before
…
1. WRITING A STEM
1.3. The stem should provide CLEAR instruction.
Weak stem: All of the following words can fit in the blank except for
…
 Strong stem: All of the following words can fit in the blank EXCEPT for
…
1. WRITING A STEM
1.4. The stem should not provide the answer.
Weak stem: He was injured in an ________ on the way to work yesterday.
traffic jam
accident
crash
Collision
 Strong stem: He was injured in a/ an _________ on the way to work yesterday.

1. WRITING A STEM
1.5. A stem should not include unnecessary information
Weak stem: All of the following sentences, which contain relative clauses, are correct EXCEPT for
The suitcase which my father bought was upstairs.
The suitcase my father bought was upstairs.
The suitcase bought by my father was upstairs.
The suitcase buying my father was upstairs.
Strong stem: All of the following sentences are correct EXCEPT for
…

To sum up, a good stem…
Should - test just ONE concept
- include most of the wording
- provide CLEAR instruction
Should not - provide the answer
- include unnecessary info.
2. WRITING A LEAD-IN
Should clearly indicate how to answer the MCQ (that students should choose from the options given)
Should preferably be a question
Should avoid negative questions for reason of ambiguity
Should present a task to the candidate
E.g. Which of the following words can fit in the blank?  task: choose one word
Should not contain absolute terms or frequency terms
2. WRITING A LEAD-IN
EXAMPLE
A weak lead-in
Which of the following can always go after a verb?
An adverb
An adjective
An infinitive
A gerund
 The most plausible answer is (a); however, students may reason that in some cases an adverb cannot follow a verb, e.g. taste when it refers to one of the 5 senses  result in confusion
EXAMPLES OF BAD LEAD-INS
Circle the best answers.
Circle the best answer.
Choose the best answer.
Choose the correct answer
Choose the letter of the best answer.
Circle the appropriate letter A – D.
EXAMPLES OF BAD LEAD-INS
Choose the correct answer to fill in the space.
Decide which answer A, B, C, or D, best fits the space.
Read the following text and answer the questions.
Read the following passage then choose the correct answer to each question.
Read the following passage and, for each question, choose the one best answer – A, B, C, or D – based on what is stated in the passage or on what can be inferred from the passage
EXAMPLES OF BAD LEAD-INS
Read the following passage and answer the questions by choosing A, B, C, or D, which you think is correct.
In this section you must choose the word or phrase which best complete each sentence. For each question, 1 to 20, indicate on your answer sheet the letter A, B, C, or D against the number of the question.
Pick out the word whose underlined part has the pronunciation different from that of the others.
EXAMPLES OF GOOD LEAD-INS
Using information from the text, answer the following questions. Write the appropriate letter A – D in boxes 13 and 14 on your answer sheet.
Choose the appropriate letter A – D and write it in boxes 15 – 17 on your answer sheet.
For question 1 – 15, read the text below and decide which word or phrase A, , C, or D best fits each space.
EXAMPLES OF GOOD LEAD-INS
Questions 1-15 are incomplete sentences. Beneath each sentence you will see four words or phrases, marked A, B, C, and D. Choose the one word or phrase that best completes the sentence. Then, on your answer sheet, find the number of the question and fill in the space that corresponds to the letter of the answer you have chosen. Fill in the space so that the letter inside the oval cannot be seen.

SAMPLE OF LEAD-INS
READING SKILL
Main idea
Inference
Restatement
Vocabulary
Negative question
Reference
Author’s Attitude, Opinion, or Purpose
Organization
Identification of Lines
Previous or Following Possible Topic
SAMPLE OF LEAD-INS
READING SKILL
Main idea
What is the main subject of the passage?
What is the main topic of the passage?
With what topic is the passage primarily concerned?
What does the passage mainly discuss?
SAMPLE OF LEAD-INS
READING SKILL
2. Inference
It can be inferred that
…
The author implies that
…
It can be implied that
…
SAMPLE OF LEAD-INS
READING SKILL
3. Restatement
According to the passage…
E.g. According to the passage, what is the monkey expecting?

SAMPLE OF LEAD-INS
READING SKILL
4. Vocabulary
In the third sentence, the phrase ________ most probably means that ________.
In the second paragraph, the author uses the phrase ________ to illustrate _____.
As used in line 6, the word _______ refers to ________.
In line 9, the word ________ could be best replaced by ___________
The word ________ in line 5 is closest in meaning to ________.
SAMPLE OF LEAD-INS
READING SKILL
5. Negative question
Which of the following is NOT mentioned in the passage?
Which of the following would NOT be an example of ________?
All of the following were mentioned in the passage EXCEPT _________

SAMPLE OF LEAD-INS
READING SKILL
6. Reference
In line X, what does the word “it” refer to?
The word “they” in the first sentence refers to _________
SAMPLE OF LEAD-INS
READING SKILL
7. Author’s attitude, opinion or purpose
What is the main purpose of this passage?
Why does the author mention ________?
Which of the following indicates the author’s attitude towards _________?
With which of the following statements would the author most likely to agree?
Which of the following would best describe the author’s tone?
SAMPLE OF LEAD-INS
READING SKILL
8. Organization
Which of the following statements best describes the organization of the passage?
How are the events of the passage presented?
Where in the passage does the author describe _________?
The ideas in the passage are divided into two paragraphs in order to contrast _________
SAMPLE OF LEAD-INS
READING SKILL

9. Identification of lines
Where in the passage can you find information about ________?
Where in the passage the author mention _________?

SAMPLE OF LEAD-INS
READING SKILL
10. Previous or following topic
Which of the following would most likely be the topic of the next paragraph?
Which of the following would most likely be the topic of the previous paragraph?
3. CHECKING THE STEM AND THE LEAD-IN
Both should be clear, precise, and simple
Do not create a “test within a test”
E.g. Choose the best definition of the word.
Photosynthesis is the process
by which green plants turn carbon dioxide and water into food using energy from sunlight
of taking foods into the body and digest it
of transforming chemical energy into thermal energy
Of pumping the blood from the heart to parts of the body.
 This does not test vocabulary alone; it tests biological knowledge at the same time.


4. WRITING THE OPTIONS
4.1. An MCQ should have only one clear answer
Weak options:
Which is the best word to fill in the blank?
John helped _________ the room.
me to decorate
decorate
decorating
me decorating
 Both (a) and (b) are correct
 Strong options:
…
me to decorate
with decorate
decorating
me decorating
 Correct answer: a
4. WRITING THE OPTIONS
4.2. Distracters should be clearly incorrect but plausible
Weak distracters:
Before ________, they used horse-drawn wooden carts.
farmers have had tractors
tractors owned by farmers
farmers had tractors
farmers owned by tractors
(d) is not a good distracter since it is logically wrong and can be easily crossed out.
Strong distracter:
…
d. farmers owning tractors
 All of the options are meaningful and grammatically correct.

4. WRITING THE OPTIONS
4.3. The options should be of equal length and similar structure
Weak options:
The present perfect simple can be used to describe an action
that starts in the past, continues to the present, and may continue in the future.
that is always true.
that refers to a future event.
that is completely finished.
 Option (a) is too long compared to the rest  It stands out as the key
Strong options:
…
that finished a short time before
…
4. WRITING THE OPTIONS
4.4. Avoid grammatical cues.
Weak options:
You can believe him. He never ______ lies.
speak
tells
gives
says
(a) is too obviously wrong  easily crossed out
Strong options:
…
speaks
…

To sum up, good options…
Contain only ONE clear answer
Contain clearly incorrect but plausible distracters
Are of equal length and written in similar structures
Exclude grammatical cues
AFTER WRITING
Do the MCQs assess important concepts?
Are the MCQs pitched at a suitable difficulty level?
Are all the options homogeneous?
Do the positions of the correct options vary?
Do the MCQs have any technical mistakes? (spelling, grammar, etc.)

For further information, please go to
http://www.englishcfl.org/forum
http://www.englishcfl.org/uploads

THANK YOU
FOR YOUR ATTENTION!