Station 01
Why
What can an exercise on screen do that paper cannot?
The objection is right
There is a widespread and well-founded scepticism about digital exercises in teaching. Multiple choice measures recognition. The student sees four statements, three of which were written to be wrong, and picks the one that sounds most like the teacher.
That can show whether a term has crossed the student’s path before. It cannot show whether the student can use it on a case they have not seen, and it cannot show at all whether they can weigh two explanations against each other, which is what most learning outcomes in higher education ask for.
In SOLO terms (Biggs & Collis), multiple choice lives at the unistructural and multistructural levels and struggles to get further. The relational level requires the student to relate one thing to another. The extended abstract level requires them to generalise beyond what they were given.
So the objection holds. But it only holds against the kind of web exercise that is a test. There is another kind, which tests nothing at all, and whose only purpose is to produce a shared experience or a shared dataset with a surprise in it, which the lesson is then about.
The good web exercise does not move the answer forward in the lesson. It moves it back, and puts something in front of it that the students could not have reasoned their way to.
The paper test
The advantage is not general, and it is worth being precise about. There are seven tools a screen has and paper does not. An exercise that uses none of them belongs on paper, and then paper is also the right choice.
POE Predict–Observe–Explain
The order can be enforced. The student cannot leaf ahead to the answer and can be made to commit to one before the answer comes. On a sheet, you can always look further down.
Example: CE conformity routes on ex.samle.dk. The routes only unlock once the device class is right.
Try CE Conformity Routes ex.samle.dk · Fra overskrift til metodeafsnit psykologi.samle.dk
White & Gunstone 1992 · Richland, Kornell & Kao 2009
A/B Split-class experiment
Half the group can get one version and half another, without anyone noticing. It is a between-subjects design in a classroom, and it costs nothing.
Example: anchoring, split in two, on psykologi.samle.dk. Half the class estimates after a high number, half after a low one, and the two distributions appear side by side on the projector.
Try Ankring, delt i to psykologi.samle.dk · Ordlydens pris psykologi.samle.dk
Kohavi, Tang & Xu 2020 · Kolb 1984
RT Reaction time
Exposure for exactly two seconds. Reaction time in milliseconds. A number of classic findings are pure timing phenomena and do not exist off the screen.
Example: Stroop with real reaction times on psykologi.samle.dk. First your own distribution, then the class’s.
Try Stroop med rigtig reaktionstid psykologi.samle.dk · Forandringsblindhed psykologi.samle.dk
Donders 1868
ARS Audience response
Thirty answers become a distribution in three seconds, while the lesson is still running. The group gets its own data to look at, and small samples and skewed selections become a problem they produced themselves.
Example: the implant heatmap on 3dklinikken.samle.dk. Every attempt to place an implant lands on the class’s shared map.
Try Kroppen 3dklinikken.samle.dk · Den norm, ingen har psykologi.samle.dk
Mazur 1997 · Kay & LeSage 2009
SRS Spaced retrieval
The page remembers what the student answered three weeks ago. The student does not. That is the difference between reading about forgetting and seeing your own forgetting curve.
Example: the forgetting curve on psykologi.samle.dk, and the group’s card on ex.samle.dk that follows them from one exercise to the next.
Try Deres egen glemselskurve psykologi.samle.dk · Intended purpose ex.samle.dk
Ebbinghaus 1885 · Cepeda et al. 2006 · Roediger & Karpicke 2006
SIM Interactive simulation
Turn a dial on a model and see what follows, as many times as you like. Paper can show one state. The screen shows the consequence of a choice.
Example: digital filters on ex.samle.dk. A heart sound through FIR and IIR filters, with four plots moving together.
Try Digitale filtre ex.samle.dk · Design undersøgelsen psykologi.samle.dk
de Jong & van Joolingen 1998 · Wieman, Adams & Perkins 2008
RWD Real-world data
Material that cannot be printed or handed out: a stack of forty CT slices, a heart-sound signal, six X-rays. The student works in what professionals work in.
Example: the segmentation room on 3dklinikken.samle.dk. Thirteen-year-olds segment a real lower leg and see their own 3D model.
Try Segmenteringsrummet 3dklinikken.samle.dk · Risikovurdering ex.samle.dk
Herrington & Oliver 2000
MCQ is not one of the seven tools. The format is familiar and supported everywhere, but the seven tools open possibilities that a multiple-choice question will never be able to hold.
What is not on the list
Automatic marking, points, badges, adaptive difficulty. None of the seven is about relieving the teacher. They are about producing something in the lesson that cannot otherwise be produced.
One of the four sites does have points and a diploma. It is for 13-year-olds, and it is a choice about motivation for that age group, not a reason for the exercises to be on a screen.
Every exercise should have one sentence about why it cannot be done on paper. If that sentence is weak, the exercise is not worth the time, and paper is cheaper.
Run the paper test on your own idea
Think of one exercise you would like to build, and answer seven questions about it.
What it does not replace
None of the exercises is a lesson. They take between three and twelve minutes, and they are worthless without the follow-up. What the group sits with afterwards, their own numbers, their own wrong guess, their own conviction, is the raw material. Teaching is still what happens when the teacher picks it up.
None of them gives a grade, and none of them should. The moment one of them starts assessing, students stop answering honestly, and then the exercise stops working.
And they are not free to make. In return they work again next year and the year after, and they can be shared with colleagues who teach the same thing.
Where GenAI makes a difference, and where it should stay out
None of the exercises needs AI to work. They are web pages that remember something, and the technology behind most of them is older than the students who use them.
Where it helps
- Building them. That is the real reason the exercises did not exist before. An exercise that used to take a week of programming now takes an evening.
- Variation. Many different cases over the same structure, instead of the one everybody has already seen. Written at build time; the students read fixed text.
- Two uses that put GenAI in front of students: feedback on free text (not a grade, but “you have not said who the participants were”) and a simulated respondent to rehearse an interview on. They can be worth it, but they are a different kind of tool, and they raise the questions on the Privacy station.
Where it should stay out
- As the assessor. A grade destroys the honesty the exercises live on.
- Between teacher and student. The exercise should deliver something into the lesson, not take it over.
- As a hidden component. If students cannot see what is a machine, there is an ethics problem.
- As the argument. If the first thing a sceptical colleague hears is “AI”, the conversation about teaching is over before it has begun.
References
- Biggs, J. B. & Collis, K. F. (1982). Evaluating the Quality of Learning: The SOLO Taxonomy. Academic Press.
- Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T. & Rohrer, D. (2006). Distributed practice in verbal recall tasks. Psychological Bulletin, 132(3), 354–380.
- de Jong, T. & van Joolingen, W. R. (1998). Scientific discovery learning with computer simulations of conceptual domains. Review of Educational Research, 68(2), 179–201.
- Donders, F. C. (1868/1969). On the speed of mental processes. Acta Psychologica, 30, 412–431.
- Ebbinghaus, H. (1885). Über das Gedächtnis. Duncker & Humblot.
- Herrington, J. & Oliver, R. (2000). An instructional design framework for authentic learning environments. Educational Technology Research and Development, 48(3), 23–48.
- Kay, R. H. & LeSage, A. (2009). Examining the benefits and challenges of using audience response systems: A review of the literature. Computers & Education, 53(3), 819–827.
- Kohavi, R., Tang, D. & Xu, Y. (2020). Trustworthy Online Controlled Experiments. Cambridge University Press.
- Kolb, D. A. (1984). Experiential Learning. Prentice Hall.
- Mazur, E. (1997). Peer Instruction: A User’s Manual. Prentice Hall.
- Richland, L. E., Kornell, N. & Kao, L. S. (2009). The pretesting effect: Do unsuccessful retrieval attempts enhance learning? Journal of Experimental Psychology: Applied, 15(3), 243–257.
- Roediger, H. L. & Karpicke, J. D. (2006). Test-enhanced learning. Psychological Science, 17(3), 249–255.
- White, R. & Gunstone, R. (1992). Probing Understanding. Falmer Press.
- Wieman, C. E., Adams, W. K. & Perkins, K. K. (2008). PhET: Simulations that enhance learning. Science, 322(5902), 682–683.