Affordances offered by e-assessment tools > Comparative judgement

Question 45
How can comparative judgement be used for e-assessment?

What motivates this question?

A new Moodle plug-in is available for CJ. This takes care of administrative issues such as how do students submit their work etc.

Perhaps the question should say summative assessment. There are decisions to make and barriers to navigate when implementing comparative judgement in HE. For example:

  • CJ claims to need collective judgement, but in HE it is common for one lecturer to mark the work for one module. Do we ignore the claim that collective judgement is needed, or do we used peer assessment? (and see Q19: How can peer assessment be used as part of e-assessment?).
  • CJ offers no feedback in the sense of red ink on students work. For some institutions and some lecturers this is a barrier.
  • CJ makes no use of rubrics, so how will students know what they are aiming for? Again this might be a barrier for some institutions.
  • On the other hand, it is possible to suggest some key issues that might be included in a “high quality script”. This guidance can be compared to rubrics. How will that affect the judgement?
  • Is it possible to also let the quality of the judging count towards the grade? One measure is the jugdes’ infit. This is a possibility to avoid judges who just click to do their minimum number of judgements.
  • The above are subject general issues. Important to consider maths specific CJ issues. For example, we would expect to use CJ to assess nebulous learning outcomes such as conceptual understanding, proof comprehension, problem solving, sustained reasoning.

What might an answer look like?

Models of CJ use with validated assessment outcomes. (Validated against: achievement data? other module tests?)

References

Jones, I., & Alcock, L. (2014). Peer assessment without assessment criteria. Studies in Higher Education, 39(10), 1774–1787. https://doi.org/10.1080/03075079.2013.821974

Jones, I., & Sirl, D. (2017). Peer assessment of mathematical understanding using comparative judgement. Nordic Studies in Mathematics Education, 22(4), 147–164.