Affordances offered by e-assessment tools > Free-form student input

Question 40
How can the suitability of e-assessment tools for summative assessment be improved by combining computer-marking and pen-marking?

Many summative assessment situations provide student with considerable freedom of expression. On the other hand, most e-assessment tools constrain students’ ability to freely express themselves, an extreme example being MCQ in which the student can only select from predefined options. It therefore seems sensible, as automatic e-assessment tools become better at assessing students’ line by line working to investigate how to combing computer-marking and pen-marking. Indeed, to some extent pen-marking may always remain necessary since subjective matters like “quality” are unlikely to ever be automated.

What motivates this question?

Rowlett (2020) investigated use of a partially-automated assessment approach, “in which automated question setting is used to generate individualised versions of a coursework assignment, which is completed by students and marked by hand”. This aims to increase the reliability of a coursework assignment, using randomisation to reduce copying and collusion, while not limiting the validity, in terms of being able to set complex and open-ended tasks with diverse submission formats. This is because the advantages of randomised question-setting are not paired with the limitations of automated marking. Rowlett finds this approach to be capable of adapting a coursework assignment to make it less sensitive to copying and collusion (and therefore more reliable) while maintaining its validity, though leading to reduced efficiency for the marker.

Blyth and Labovic (2009) use assessment which is randomised by students themselves via Maple for submission and marking by hand.

Approaches are used in statistics where a set of static questions are combined with a randomised data set, producing individualised work that is marked by hand (for example, see Hunt, 2007 and Fawcett, Foster and Youd, 2008). The approach of Fawcett, Foster and Youd includes an e-assessment system and use of multiple-choice questions for some parts.

Sangwin (2015) proposes combined assessments, in which a “routine calculation within a longer proof” is “checked automatically… before the whole piece of work is submitted to an intelligent human marker”.

Such approaches are largely experimental, but show promise. Key to them is the ability to access some advantages of e-assessment, while avoiding some of the limitations.

What might an answer look like?

This is a wide and rather vague question, and so answers are likely to occur only to more specific sub-questions. Initially we might seek answers to interface design, letting students provide partial answers which can be automatically assessed alongside other work needing human attention.

A thesis currently under embago has recently been completed investigating the affordances of the Graide software. https://www.6bit.co.uk/.

References

Blyth, B., & Labovic, A. (2009). Using Maple to implement eLearning integrated with computer aided assessment. International Journal of Mathematical Education in Science and Technology, 40(7), 975–988, https://doi.org/10.1080/00207390903226856

Fawcett, L., Foster, B., & Youd, A. (2008). Using computer based assessments in a large statistics service course. MSOR Connections, 8(3), 45–48, https://doi.org/10.11120/msor.2008.08030045

Hunt, N. (2007). Individualized Statistics Coursework Using Spreadsheets. Teaching Statistics, 29(2), 38–43, https://doi.org/10.1111/j.1467-9639.2007.00254.x

Rowlett, P. (2020). Partially-automated individualized assessment of higher education mathematics. International Journal of Mathematical Education in Science and Technology. In press. https://doi.org/10.1080/0020739X.2020.1822554

Sangwin, C. (2015). Computer Aided Assessment of Mathematics Using STACK. In S.J. Cho (Ed.), Selected Regular Lectures from the 12th International Congress on Mathematical Education (pp. 698–713). Cham, Switzerland: Springer, https://doi.org/10.1007/978-3-319-17187-6_39