Writing better MCQs

Multiple choice questions (MCQs) have a bit of an image problem: they’re often seen as simply a testing tool (and not a very good one at that because they’re easily outsmarted) as well as only being suitable for lower-order thinking tasks.

We can use MCQs for most of Bloom's cognitive levels

We can use MCQs for most of Bloom's cognitive levels

While this this is often true, it is not actually the fault of the question type!

Well-written MCQs are an effective way to challenge students at all of Bloom’s cognitive levels (except for ‘Create’).

It’s all to do with instructional design:

  1. how the questions and responses are worded
  2. what they test and
  3. at which point of the learning process the question and feedback are presented

In this post, we'll share our favourite tips for writing better multiple choice questions:

Tip #1 – Use automated feedback

Tip #2 – Don’t make them too easy!

Tip #3 – Use previous students’ responses to written-answer tasks to find common misconceptions

Tip #4 – Combine two MCQs for analytical thinking

Tip #5 – Set tasks that require more than one correct option

However, before we dive in and look at how we can write better questions, let's take a look at the parts that make up MCQs. This will come in handy later when we get to the tips.

Anatomy of a MCQ

A multiple choice question consists of two parts: the stem and the responses. The stem is the question text and the responses are the various answer options students can choose. Responses are further broken down into the correct answer and distractors: logical and plausible misconceptions of the best answer.

If you think that the above is not a very good example of a MCQ, you’re absolutely right! We’ll get to that soon, but first let’s dispel the myth that MCQs are just a testing tool:

Tip #1 – Use automated feedback

We all want to give more and better feedback. Wouldn’t it be great if we could:

  1. clone ourselves for a lesson
  2. give each student personalised, real-time feedback on any of their responses 
  3. address misconceptions without delay
  4. not actually spend time doing any of that?

Welcome to automated feedback!

Stile lets you add this as you’re writing the question. Writing them may take a little longer initially, but now suddenly our MCQs are a teaching tool that not only does its own marking but also gives targeted, personalised and timely feedback! To find out more about what kind of feedback helps our students learn best, check out this post.

Here's one example of automated feedback added to an evaluative question:

 

Tip #2 – Don’t make them too easy!

Students will try to answer multiple choice questions without actually knowing the answer, purely by going on other cues, such as:

1.     clearly implausible distractors: these often include extreme words like ‘only’, ‘always’ or ‘never'

2.     three short distractors and one long response: the long response is usually the correct answer

3.     all of the above’: students can recognise that more than one option is correct without understanding the rest

4.     none of the above’: this doesn’t test what students know — they’re only able to show that the correct response is missing

5.     overlap between responses: making all responses mutually exclusive and as unambiguous as possible will prevent them from simply matching up the overlapping parts and finding the answer by deduction rather than knowledge

6.     giving away the ‘password’: If one response contains a key term (e.g. ‘because of Newton’s second law’), students don’t need to understand its meaning to recognise its probable correctness. Paraphrase Newton’s second law instead.

Avoiding these wherever possible should ensure I'm testing students’ subject knowledge rather than their ability to 'game' poorly written questions. 

Here's a good example of how not to do it! Vague language in the stem (‘typically’), non-parallel responses, extreme language and all/none of the above!

 

Tip #3 – Use previous students’ responses to written-answer tasks to find common misconceptions

Writing convincing distractors is probably the hardest part of writing a good MCQ. The correct answer is right in front of our students’ eyes; they should be able to find it, but it shouldn’t leap out at them immediately.

If you’ve been teaching for a while, you might already have everything you need, though — courtesy of your students! Simply go through old assessments that included written-answer questions and look for common misconceptions in student answers. The three most common ones in a question would make for excellent distractor candidates!

If you’re new to teaching, you could ask a friendly colleague for some data.

Tip #4 – Combine two MCQs for analytical thinking

While it's more challenging to write questions that develop and test higher-order thinking, it's certainly possible, as long as there's sufficient context (which can be provided in the stem or next to the question). 

Linking words such as ‘because’ and ‘unless’ get students really thinking about their choices. Here is one example:

 

Tip #5 – Set tasks to require more than one correct option

It’s relatively easy for students to pick one out of four options by chance. However, requiring more than one correct answer makes guessing very difficult. ‘Which of the following are examples of…’ is all that’s required to make students really think about each option. You can meet them in the middle by telling them how many they need to pick or leave it open to make it a real challenge.

Summary

So here they are again:

  1. Automated feedback turns MCQs into a teaching tool that provides instant, personalised feedback for each student
  2. Challenge your students by avoiding dead giveaways
  3. Dig into past student work to discover the most common student misconceptions from written-answer responses
  4. Combine two MCQs to discover students’ thinking processes and develop their analytical thinking skills
  5. Require more than one correct answer to minimise lucky guesses and getting students to carefully consider each option.

We hope this will help you write better multiple choice questions or that it's inspired you to try writing some if you previously shied away from them.

 

Sources

We used this slide deck by Carol A. Kominski at UNT Health Science Center, this fantastic toolkit by the John Hopkins Bloomberg School of Public Health as well as these sample questions by Kimberley Green of Washington State University as sources for the material presented in this post. This guide by Vanderbilt University's for Teaching  is great too.