Assessing students online in the post-information age

In this post, I'll share three tips on how to run online assessments - but first, let's take a look at the current landscape of education and assessment in general.

Conducting an online assessment is kind of like an open book assessment on steroids: back when kids could bring in their books they wouldn't be able to look up everything in the time they had without knowing their stuff: search doesn't work on printed works! 

With the advent of Google, Wikipedia, eBooks and the internet in general, this has of course completely changed. And by necessity, so has the role of the teacher and education at large, as knowledge transfer is becoming increasingly irrelevant in the post-information age. Then again, this isn't news. Seymour Papert told us this well before he passed away in July:

Many other educators have since followed suit. When students can just google what any given concept means, with essays written by experts on the topic freely available in just a few clicks, we need to not only change the way we teach but also how we assess.

Before I go into tips on how to assess, here's a word of warning from one of my favourite educators, Alfie Kohn:

Never grade students while they are still learning something and, even more important, do not reward them for their performance at that point. Studies suggest that rewards are most destructive when given for skills still being honed (Condry and Chambers 1978). If it is unclear whether students feel ready to demonstrate what they know, there is an easy way to find out: ask them.
— Alfie Kohn, http://www.alfiekohn.org/article/grading/

Let's leave the (summative or marked) assessment for until after the learning is done wherever possible. Turning every lesson into an assessment with marks is not helping students learn - all evidence seems to point in the opposite direction. Continuous, formative assessment, on the other hand, works amazingly well - just leave the marks out and replace them with feedback!

So without further ado, here are my three main tips for online assessment are:

  1. Change the context – test for understanding of principles, not recalling facts or knowledge
  2. Ask non-Googleable questions: if the answer can be found in a quick Google search, it probably doesn’t need to be asked in the first place!
  3. Hone skills you want students to develop: create assessments that focus on students’ creative problem solving and finding ability.

1. Change the context

Memorising large amounts of seemingly disconnected facts is something that I wish I’d done a lot less of during my school days. Facts are quickly forgotten, especially if there is little or no explicit focus on the underlying principles connecting them. How much do you remember from your own school days?

Therefore, we need to design assessments around things that we definitely want students to remember or be able to accomplish in the future: the key principles.

Let’s say I’m teaching a science lesson on resources and want my students to be able to distinguish natural from synthetic resources. Memorising a large number of each would be possible, but not very practical. Focusing on the principle and then changing the context in which something is learned, on the other hand, is much more effective.

It’s important that this already happens during the learning process in formative assessments. That way, students have the opportunity to examine a concept in a variety of contexts before they get to the assessment stage.

Here’s an example from the Resources unit in Cosmos Lessons. This is a question that students answer after having watched a video (which gave crude oil and plastic as examples) and played with a simulation on synthetic and natural resources, the ability to distinguish between the two being the learning goal for this lesson. It acts as a concept checking exercise:

After the lesson, they get a quick pop quiz on the same topic to reaffirm the concept in the same context, but phrased differently, explicitly tied to the stated learning goal:

At the end of the unit, the summative assessment introduces a completely new resource — soda-lime glass — that hadn't been mentioned at any point in the unit:

This ensures students can apply the newly-learnt concept to materials that they've potentially never heard of. It also happens after two rounds of formative assessment.

2. Ask non-Googleable questions

Categories in the cognitive domain of Bloom's taxonomy

Categories in the cognitive domain of Bloom's taxonomy

The three questions above are still on the lower rungs of Bloom’s pyramid: Remember for the first two (crude oil and plastic had already been explicitly mentioned in the video) and Understand for the last one (applying the understanding to a different context).

In order for students to retain this ability to differentiate two concepts, they need some way to apply it – simply changing contexts won’t be enough. We need to move up in Bloom’s pyramid.

If a maths teacher spent half an hour with her class on calculating the volume of various coffee mugs, she’d expect students to be able to calculate the volume of any cylindrical object for example. However, there is no longer a need to remember the formula, as a Google search for ‘volume of cylinder’ not only brings up the formula itself but even an interactive calculator that does all the work for you:

It even lets you solve for surface area, radius and more. It’s quite amazing, actually – give it a go if you haven’t tried it before.

The fact that this is now no longer required for students to memorise opens up quite a few possibilities. We can create more creative assessments that require students to be more analytical, decide which formula to use in which situation and generally be more rigorous. The assessment examples can be more authentic and relevant.

To continue with the example above, rather than give students the variables to plug in as measurements, give them an object and they need to decide what is what. This TEDx talk is a great example for this if you haven’t seen it already:

3. Hone skills you want students to develop

Most of the knowledge students acquire over their time at school will be lost by the time they graduate, and that’s completely normal. However, even if the explicit knowledge is lost, they will generally still be able to remember where it fits into the greater scheme of things, like a placeholder.

If their search skills are well-developed, they can fill these knowledge gaps quickly and easily, just like I do every day at least a dozen times. The ability to fill knowledge gaps quickly and effectively is now more important than having the knowledge in the first place, with a few notable exceptions, of course – mid-surgery Google searches are currently not an option! That said, this could change rapidly with the advent of augmented reality.

Therefore, we need to create assessments that focus on such skills, like students’ problem solving and finding ability. Andreas Schleicher, – Head of the OECD, the body that runs the PISA tests – agrees:

We look very carefully at how the world and the skills that people need are changing and then we try to reflect that in our measure.

For example, we know that it is important for people to collaborate, compete, connect, work with each other. In 2015, in the latest PISA, we have added one domain called collaborative problem solving.
— Andreas Schleicher, EdSurge interview, April 2016

If at all possible, design your assessments with this in mind – and yes, I realise how difficult this is!

At the end of the day, these will be different depending on which subject(s) you teach, yet also include cross-disciplinary skills to bring together these skills.

How are you assessing your students?

What kind of questions do you write to ensure tasks help students develop and retain skills that will help them become successful in later life?

We'd love to hear from you in the comments or on Twitter.

Guido Gautsch
VP of Happiness 😊