Modified who partograph ppt presentation

Writing MCQs

Susan Chamberlain
Dept of OBS / GYN
Queens University

Writing MCQs
At the end of this session, the participant will
be able to
Identify different types of MCQ items
Describe the structure of an MCQ
List the technical flaws common to MCQs
write one well-structured MCQ testing
appropriate content

Writing MCQs
At the end of this session, the participant will
be able to
Identify different types of MCQ items
Describe the structure of an MCQ
List the technical flaws common to MCQs
write one well-structured MCQ testing
appropriate content
All of the above

MCQ check-list
Why we write them
What type is best
What flaws are to be avoided
What content to test
How to construct it

Why do we test?
Communicate to the students what material
is important (formative vs summative)
Motivate students to study
Identify areas of deficiency in need of
remediation or further learning
Determine final grades or make promotion
decisions
Identify areas where the course / curriculum
is weak

What should be tested?


Exam content should match course
objectives
Important topics should be weighted more
heavily than less important topics
The testing time devoted to each topic
should reflect the relative importance of the
topics
The sample of items should be
representative of the instructional goals

What should be tested?

Sampling
Performance on the sample provides a basis
for estimating achievement in the broader
domain that is actually of interest.

Questions sample:
TOPICS
SKILLS ie determining diagnosis, next step in
management, etc.

MCQ items
Should test important information either
common or life-threatening
Should be well structured

Flaws benefit test-wise examinees


Irrelevant difficulty does not test examinees
specific knowledge

Anatomy of an MCQ Item


Stem
Lead-in
Options

One correct answer


Distractors

Types of MCQs
True / False family
Assess recall of isolated facts
Stems must be clear and unambiguous
Options must be absolutely true or false or
examinee must decide how true options are

Generally not recommended usually have


subtle flaws

Types of MCQs: True / False


Which of the following statements is true regarding contraception?
A) oral contraceptives should not be initiated until six weeks after
therapeutic abortion to avoid the risk of hypercoagulability
B) an intrauterine device can provide good contraceptive
protection 5 years after insertion
C) oral contraceptives should not be prescribed to individuals
under the age of 15 without parental consent
D) lesbian women do not require counseling about contraception
or sexually transmitted diseases
E) oral contraceptives should not be prescribed for women over
the age of 35

Types of MCQs
One-best-answer family (A type)
Options may be diagrammed:

D C A

Least correct

Most correct

Correct answer is the most likely correct


Better application of knowledge, integration,
synthesis and judgment

Types of MCQs:
One-best-answer family
A 24 year old university student came to see you because she is
missing several days of school each month because of severe
dysmenorrhea unresponsive to the OCP and NSAIDS. So you
complains of deep dyspareunia and has a tender uterosacral
module on the pelvic examination. Which one of the following
conditions is most likely responsible for her symptoms?
A) Chronic pelvic inflammatory disease
B) endometriosis
C) adenomyosis
D) fibroids

MCQ check list
Why we write them
What type is best
What flaws are to be avoided
What content to test
How to construct it

MCQ Technical Flaws


Issues Related to Test-Wiseness
Issues Related to Irrelevant Difficulty

MCQ Technical Flaws

Issues Related to Test-Wiseness


provide clues to the right answer
examinee can eliminate options based on flaws
and not on knowledge

Issues Related to Irrelevant Difficulty

Item Flaws: Test-wiseness


Grammatical cues
Logical cues
Absolute terms
Long correct answer
Word repeats
Convergence strategy

Item Flaws: Test-wiseness


Grammatical cues
One or more distractors don't follow
grammatically from the stem

Item Flaws: Test-wiseness


Grammatical cues
A 24 year old patient presents with a one year history
of dysmenorrhea, mid-cycle pain, dyspareunia,
and painful bowel movements with her periods.
The most likely diagnosis is:
A)
B)
C)
D)

Fibroid uterus
Adenomyosis
Endometriosis
Psychosomatic condition

Item Flaws: Test-wiseness


Logical cues
A subset of the options are collectively
exhaustive

Item Flaws: Test-wiseness


Logical cues
Arterial blood gases in a normal pregnant
woman, compared to her non-pregnant state may
show which of the following?
A) Decreased pO2
B) Increased pCO2
C) Decreased pCO2
D) Decreased pH
E) No difference in pCO2

Item Flaws: Test-wiseness


Logical cues
One or more distractors don't follow
logically from the stem

Item Flaws: Test-wiseness


Logical cues
An important tool for assessing fetal well-being is
the non stress test. A well baby demonstrates
the following fetal heart rate pattern:
A)
B)
C)
D)
E)

A baseline heart rate above 120 bpm


Evidence of fetal movement
Baseline variability
Two fetal heart rate accelerations in 20 minutes
Fetal breathing movements

Item Flaws: Test-wiseness


Absolute terms
Always, never
Not a problem if stem is long and lead-in
asks a precise question

Item Flaws: Test-wiseness


Absolute terms
In patients with a previous Cesarean section:
A) Vaginal birth is NEVER an option
B) Repeat Cesarean section is less dangerous to the
mother
C) The uterine scar may rupture during labor
D) Care should ALWAYS be supervised by an
obstetrician
E) No further pregnancies should EVER be considered

Item Flaws: Test-wiseness


Long correct answer
Correct answer is longer, more specific, or
more complete than other options

Item Flaws: Test-wiseness


Long correct answer
Which of the following problems in vaginal
breech delivery is most likely to cause serious
long-term complications?
A)
B)
C)
D)

Failure to descend
Cord prolapse
Nuchal arms
Entrapment of the after coming head with an
incompletely dilated cervix
E) 1 minute Apgar score of 4

Item Flaws: Test-wiseness


Word repeats
A word or phase is included in the stem and
in the correct answer

Item Flaws: Test-wiseness


Word repeats
A 24 year old with a previous history of chronic
PID presents with primary infertility. Select the
procedure that would be most appropriate to
evaluate the status of her tubes and pelvic
anatomy
A)
B)
C)

Post-coital test
Endometrial biopsy
Laparoscopy with dye insufflation of the fallopian
tubes
D) hysterosalpingogram

Item Flaws: Test-wiseness


Convergence strategy
The correct answer includes the most
elements in common with the other options

Item Flaws: Test-wiseness


Convergence strategy
Normal bladder neck mobility is measured by:
A) A Q-tip angle measurement under 40 degrees
B) A Q-tip angle measurement under 30 degrees
C) A bladder neck ultrasound angle less than 30
degrees
D) A Q-tip angle measurement under 20 degrees

MCQ Technical Flaws

Issues Related to Test-Wiseness

Issues Related to Irrelevant Difficulty


do not assess knowledge of intended topic

Irrelevant Difficulty

Options are long, complicated or double


Extraneous details in stem
Numeric data not stated consistently
Vague frequency terms
Non-parallel language in options
Stems are tricky or unnecessarily complicated
None of the above

Irrelevant Difficulty
Options are long, complicated or double
Extraneous details in stem
Shifts difficulty from content knowledge to
speed reading

Irrelevant Difficulty
Options are long, complicated or double
Mrs. Brown is a 70 year old woman who had a stroke 6 months
ago and is still impaired. Her daughter tells you that her
mothers recent urinary incontinence makes her care
unmanageable. She is seeking permanent placement. Mrs. Brown
complains of urinary frequency, nocturia, and urge incontinence.
Which of the following statements is true?
A)
B)
C)
D)
E)

Neurological conditions are involved in the majority of women with


symptoms like those presented by Mrs. Brown
Diabetes has a greater impact on quality of life than Mrs. Browns
condition.
The pharmacological blockage of muscarinic receptors in the detrusor
muscle controls symptoms compatible with Mrs. Browns condition
Overactive bladder is defined as urinary incontinence accompanied by or
immediately preceded by urgency
Alpha stimulants are effective in treating this condition

Irrelevant Difficulty
Extraneous details in stem
You see in your office a 24 year old woman for her routine postpartum
visit, at 6 weeks. Her baby is irritable and cries all the time (including
during her visit.) As she is changing the diaper during her visit, you
notice that the child has a very red bum but upon questioning, she tells
you she did not notice anything abnormal. Her partner is away in
Afghanistan and she does not have a lot of friends, as they just recently
were posted in Kingston. She states that she has a hard time getting
dressed in the morning, cries easily and feels like she does not love her
child. She denies any medical or psychiatric history. Her exam is
essentially normal. She weighs 2 kg less than at her first antenatal visit
at 10 weeks. You most likely clinical diagnosis is:
A) Postpartum hypothyroidism
B) Hyperthyroidism
C) Postpartum blues
D) Postpartum depression
E) Psychotic depression

Irrelevant Difficulty
Numeric Data
Not stated consistently
Mixed format
Illogical order

Irrelevant Difficulty
Numeric data not stated consistently
Following a second episode of salpingitis, what
is the likelihood that a woman is infertile?
A)
B)
C)
D)
E)

Less than 20%


20 30 %
Greater than 50%
90%
75%

Irrelevant Difficulty
Vague frequency terms
Often, usually, rarely, frequently
Not consistently defined
* may be used in stem if otherwise carefully
constructed unambiguous question

Irrelevant Difficulty
Vague frequency terms
Gonorrhea is
A)
B)
C)
D)
E)

sometimes asymptomatic
treated with cefixime 450 mg orally
caused by a gram positive intracellular diplococcus
diagnosed by a DNA test
usually associated with vulvar ulcers

Irrelevant Difficulty
Non-parallel language in options
Non-logical order

Irrelevant Difficulty
Non-parallel language in options
In a vaccine trial, 200 2-year-old boys were given a vaccine
against a certain disease and then monitored for five
years for occurrence of the disease. Of this group, 85%
never contracted the disease. Which of the following
statements concerning these results is correct?
A) No conclusion can be drawn, since no follow-up was
made of non-vaccinated children
B) The number of cases (ie 30 cases over five year) is
too small for statistically meaningful conclusions
C) No conclusions can be drawn because the trial
involved only boys
D) Vaccine efficacy (%) is calculated as 85-15 / 100

Irrelevant Difficulty
None of the above
Problematic where judgment is involved or
if options not absolutely true or false
Turns question into true / false question
Confuses very knowledgeable students

Irrelevant Difficulty
None of the above
The main reason that fertility is reduced in a
woman in her early 40s is:
A) Irregular cycles make it hard to time intercourse
B) Chromosomal damage adversely affects oocytes
genetic complement
C) Male erectile difficulties increase
D) Hormones imbalance leads to poor endometrial
development
E) None of the above

Irrelevant Difficulty
Stems are tricky or unnecessarily complicated
Arrange the parents of the following children with Downs syndrome
in order of the highest to lowest risk of recurrence. Assume that the
maternal age in all cases is 22 years and that a subsequent pregnancy
occurs within 5 years. The karyotypes of the daughters are:
I: 46, XX, 14, + T (14q21q) pat
II: 46, XX, 14, + T (14q21q) de novo
III: 46, XX, 14, + T (14q21q) mat
IV: 46, XX, 21, + T (14q21q) pat
V: 46, XX, 21, + T (21q21q) (parents not typed)
A) III, IV, I, V, II
B) IV, III, V, 1, II
C) III, I, IV, V, II
D) I, V, III, I, V, II
E) III, IV, I, II, V

MCQ check-list
Why we write them
What type is best
What flaws are to be avoided
What content to test
How to construct it

What to test?
Items classified by cognitive processes required to
answer the question:
Recall (memory) tests knowledge of isolated facts
Interpretation (comprehension) review information
to reach conclusion, eg diagnosis
Problem solving (Reasoning) situation requires
action, eg next step in management

Process depends on the background of the trainee

What to test?
Simpler classification based on task:
Application of knowledge item **
Reach conclusions
Make predictions
Select course of action

Recall item
Test knowledge of isolated facts

What to test?

Test application of knowledge using clinical


vignettes

Adds face-validity
Usually selects more important, less trivial
Identifies student whove memorized but do not
understand

BUT experience in patient care should not


be necessary

What to test - summary

Focus item on key concepts and principles that are


essential information (without access to
references) for all examinees to understand
Test material that is relevant to learning in clinical
clerkships and beyond
Avoid items that only require recall of isolated
facts
Avoid esoteric or interesting topics that are not
essential

MCQ check list
Why we write them
What type is best
What flaws are to be avoided
What content to test
How to construct it

Writing the questions


Construct stem include all necessary
information to arrive at the right answer
Choose lead-in pose a clear question
Choose distractors

Stem *
Usually a clinical scenario
Clear, unambiguous
Should be long relative to options
Includes all relevant information

Patients age / gender
Clinical setting
Complaint
Other important info history, physical
findings, test results

Lead-in *
Follows stem
In the form of a question
Should relate to the stem
Should be answerable without looking at
options

Cover the options test

Distractors
Each should be selected by some, therefore
all plausible, none obviously incorrect
3 or 4 choices

Common misconceptions
Faulty reasoning

Distractors

Homogeneous in content to the correct answer


Should be in the same category as correct answer
diagnosis, treatment, diagnostic test
Clearly incorrect or inferior to the correct answer
Plausible and attractive to the uninformed
Similar to the correct answer in construction and
length
Grammatically consistent and logically compatible
with the stem

General guidelines for item


Construction

Include as much of the item as possible in the


stems: Long stems, short options
Avoid negatively phrased items (except or not in
the lead-in). If you must, use only with short
options
Avoid writing question of the form:
Which of the following statements is correct?
Each of the following statements is correct EXCEPT
Unfocussed
Heterogeneous options

Top-down versus bottom-up


Top down
Knowledge or recall question
Cite a disease and then ask what patient
findings are expected
Structured like textbooks
Clinically backward

Top-down versus bottom-up


Bottom up
Application of knowledge
Gives findings and asks examine to indicate
the disease
Examinees need to be able to synthesize
information from several pages of a textbook
(or notes) to answer these questions
Clinically realistic

Guidelines for Clinical Sciences


ie Clerkship and beyond
Test application of knowledge using clinical
vignettes to pose medical decisions in
patient care situations
Focus items on common or potentially
catastrophic problems
Pose clinical decisions that would be
expected of a successful examinee
Avoid clinical situations that would be
handled by a (sub) specialist

MCQ check-list
Why we write them
What type is best
What flaws are to be avoided
What content to test
How to construct it

Basic Rules for MCQs (A type) *


Focus item on important concept
Item should assess application of
knowledge, not recall of an isolated fact
Stem must pose clear question Cover the
options test
All distractors should be homogeneous
Avoid technical item flaws

Reference
Constructing Written Test Questions For the
Basic and Clinical Sciences
Case & Swanson, NBME 1996