Affordances offered by e-assessment tools > Capabilities of e-assessment

Question 35
What types of reasoning are required to complete current e-assessments?

What motivates this question?

Jonsson et al. (2014) have shown that the type of tasks assigned can affect students’ learning. Therefore it is reasonable to study the types of tasks assigned in university level courses, and it seems that very little work has been done in the area of online assessment.

Relevant prior work, focused on paper-based tasks, includes:

  • Lithner’s (2008) framework for classifying the types of reasoning required by mathematical tasks. His framework has two types of reasoning - imitative and creative. This framework has been used by various researchers to study the types of tasks in university level textbooks, assignments, and exams (Berqvist, 2012, Mac an Bhaird et al., 2017).
  • The Levels of Cognitive Demand framework of Stein and Smith (1998)
  • The framework developed by Pointon and Sangwin (2003), which was used to analyse the content of first-year exams at one university.
  • The MATH taxonomy (Smith et al., 1996), which has been used to compare the content of school-level exams and university exams (e.g. Darlington, 2014) and also to analyse the content of assessments in university modules in depth (e.g. Kinnear et al., 2020).
  • Tallman et al.’s (2016) framework for analysing the content of calculus exams.

What might an answer look like?

An analysis could be carried out on a range of online assessments. The tasks could be classified using one of the frameworks mentioned above. A comparison to textbook tasks and/or traditional assessment tasks could be made. Such an approach has been used to consider traditional examinations, and to compare different examination papers e.g. Darlington (2014).

References

Bergqvist, E. (2012). University mathematics teachers’ views on the required reasoning in calculus exams. The Mathematics Enthusiast, 9(3), 371–407.

Darlington, E. (2014). Contrasts in mathematical challenges in A-level Mathematics and Further Mathematics, and undergraduate mathematics examinations. Teaching Mathematics and its Applications, 33(2), 213-229. doi:10.1093/teamat/hru021},

Jonsson, B., Norqvist, M., Liljekvist, Y., & Lithner, J. (2014). Learning mathematics through algorithmic and creative reasoning. The Journal of Mathematical Behavior, 36, 20–32.

Kinnear, G., Bennett, M., Binnie, R., Bolt, R., & Zheng, Y. (2020). Reliable application of the MATH taxonomy sheds light on assessment practices. Teaching Mathematics and Its Applications: International Journal of the IMA, 1–15. https://doi.org/10.1093/teamat/hrz017

Lithner, J. (2008). A research framework for creative and imitative reasoning, Educational Studies in Mathematics, 67(3), 255-276.

Mac an Bhaird, C., Nolan, B., O’Shea, A & Pfeiffer, K. (2017) A study of creative reasoning opportunities in assessments in undergraduate calculus courses, Research in Mathematics Education, 19:2, 147-162.

Pointon, A.; Sangwin, C. J. (2003) An analysis of undergraduate core material in the light of hand held computer algebra systems. International Journal of Mathematical Education in Science and Technology, Vol. 34, 671-686.

Smith, G., Wood, L., Coupland, M., Stephenson, B., Crawford, K., & Ball, G. (1996). Constructing mathematical examinations to assess a range of knowledge and skills. International Journal of Mathematical Education in Science and Technology, 27(1), 65–77. https://doi.org/10.1080/0020739960270109

Stein, M. K. and M. S. Smith. (1998). Selecting and creating mathematical tasks: From research to practice. Mathematics Teaching in the Middle School, 3(5), 344-350.

Tallman, M. A., Carlson, M. P., Bressoud, D. M., Pearson, M., & Org, P. (2016). A Characterization of Calculus I Final Exams in U.S. Colleges and Universities. International Journal of Research in Undergraduate Mathematics Education, 2, 105–133. https://doi.org/10.1007/s40753-015-0023-9