Mathematical skills > Example generation

Question 54
To what extent can e-assessments meaningfully judge student responses to example generation tasks?

Some CAA systems appear well-placed to facilitate example generation tasks, as they can make use of sophisticated computer algebra systems to check the properties of student examples and give feedback - however this relies on the properties being ones that the system can evaluate.

What motivates this question?

Learner-generated example tasks have been suggested as an effective way to encourage students to engage with new concepts (Watson & Mason, 2006). Checking students’ answers and giving feedback can be a difficult task for teachers, particularly with large groups of students. Thus, e-assessment could play a powerful role in providing students with formative feedback on their examples (Sangwin, 2003), or in providing teachers with information about students’ learning that they can act on (Olsher et al., 2016). As a result, this is potentially an area where e-assessment, which is usually viewed as promoting procedural learning, can be used to assess conceptual understanding more efficiently than non-automated methods (Sangwin, 2015). For a discussion of procedural and conceptual assessment tasks, see e.g. Pointon and Sangwin (2003).

However, practical experience with creating questions of this type shows that it can often be difficult to specify a test that the e-assessment system should apply to check the properties of the student examples. For instance, if asking for functions which tend to 0 at infinity, the computer algebra system would need to have the ability to evaluate limits of arbitrary functions.

What might an answer look like?

This would involve a mix of design work to identify suitable topics and questions, and empirical work to gather typical student responses. For instance, in the limits example: while it may not technically be possible for the computer algebra system to evaluate the limit of an arbitrary function, it is likely that in practice students will only give a range of reasonably well-behaved examples which may remain within the ability of computer algebra system to evaluate. Could the computer algebra system recognise when it cannot judge an answer, to possibly refer to a teacher?

An output could be a set of exemplar questions resulting from cycles of design research (Swan, 2014). In addition, there could be an audit of different topics where current computer algebra systems struggle to deal with typical student answers.

References

Olsher, S., Yerushalmy, M., & Chazan, D. (2016). How might the use of technology in formative assessment support changes in mathematics teaching? For the Learning of Mathematics, 36(3), 11–18.

Pointon, A. & Sangwin, C.J. (2003). An analysis of undergraduate core material in the light of hand-held computer algebra systems. International Journal of Mathematical Education in Science and Technology, 34(5), 671-686. https://doi.org/10.1080/0020739031000148930

Sangwin, C. J. (2003). New opportunities for encouraging higher level mathematical learning by creative use of emerging computer aided assessment. International Journal of Mathematical Education in Science and Technology, 34(6), 813–829. https://doi.org/10.1080/00207390310001595474

Sangwin, C. (2015). Computer Aided Assessment of Mathematics Using STACK. In S.J. Cho (Ed.), Selected Regular Lectures from the 12th International Congress on Mathematical Education (pp. 698-713). Cham: Springer. https://doi.org/10.1007/978-3-319-17187-6_39

Swan M. (2014) Design Research in Mathematics Education. In: Lerman S. (eds) Encyclopedia of Mathematics Education. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-4978-8_180

Watson, A., & Mason, J. (2006). Mathematics as a Constructive Activity. Routledge. https://doi.org/10.4324/9781410613714