Design and implementation choices > Task design principles

Question 21
What design methodologies and principles are used by e-assessment task designers?

• Ian Jones
• Niclas Larson

This question is about the approaches used by e-assessment task designers, particularly regarding the design of feedback.

What motivates this question?

For design methodology, the motivation here is to uncover different practices that may be used by task designers. For instance, detailed post-hoc analysis of student responses enabled by e-assessment have been highlighted as “useful to the teacher for improving and validating feedback” (Sangwin, 2013, p. 35) – but is such analysis and improvement carried out in practice by task designers, and if so, how?

As for design principles, these are often implicit in task designers’ practice — a motivation for this question is to make them explicit. Examples of design principles being made explicit in previous work include Sangwin’s (2013, Chapter 3) description of general principles of assessment design, and Kinnear et al.’s (2021) account of the specific principles that informed the design of an online course built around e-assessment. The question highlights aspects such as timing and fading of feedback, echoing a review of research on formative feedback (Shute, 2008). These findings from research may inform the principles used by task designers.

What might an answer look like?

This seems like a scoping study. The answer to this question might involve a systematic review of the literature, a survey of e-assessment designers, and/or an analysis of existing e-assessments.

Related questions

• This question only asks about what methodologies and principles are in use; another question asks: Q22: What principles should inform the design of e-assessment tasks?
• Principles relating to feedback are addressed by various questions, e.g.
  • Q3: What are the approaches to detecting and feeding back on students’ errors?
  • Q10: Under what circumstances is diagnosing errors worth the extra effort, as compared with generally addressing errors known to be typical?

References

Kinnear, G., Wood, A. K., & Gratwick, R. (2021). Designing and evaluating an online course to support transition to university mathematics. International Journal of Mathematical Education in Science and Technology. https://doi.org/10.1080/0020739X.2021.1962554

Sangwin, C. J. (2013). Computer aided assessment of mathematics. Oxford University Press. Retrieved from https://doi.org/10.1093/acprof:oso/9780199660353.003.0003

Shute, V. J. (2008). Focus on Formative Feedback. Review of Educational Research, 78(1), 153–189. https://doi.org/10.3102/0034654307313795