Comparative judgement and the hierarchy of students’ choice criteria

2021 ◽  
pp. 1-21
Author(s):  
Jennifer Palisse ◽  
Deborah Martina King ◽  
Mark MacLean
Keyword(s):  
Comparative Judgement

scholarly journals Comparative Judgement for Linking Two Existing Scales

2021 ◽  
Vol 6 ◽  
Author(s):  
Tom Benton
Keyword(s):  
Test Equating ◽  
Comparative Judgement ◽  
Test Linking ◽  
Educational Assessments ◽  
Alternative Approaches ◽  
Time Required

This article describes an efficient way of using comparative judgement to calibrate scores from different educational assessments against one another (a task often referred to as test linking or equating). The context is distinct from other applications of comparative judgement as there is no need to create a new achievement scale using a Bradley-Terry model (or similar). The proposed method takes advantage of this fact to include evidence from the largest possible number of examples of students’ performances on the separate assessments whilst keeping the amount of time required from expert judges as low as possible. The paper describes the method and shows, via simulation, how it achieves greater accuracy than alternative approaches to the use of comparative judgement for test equating or linking.

Investigating the Comparability of Examination Difficulty Using Comparative Judgement and Rasch Modelling

2018 ◽  
Vol 18 (4) ◽  
pp. 366-391
Author(s):  
Stephen D. Holmes ◽  
Michelle Meadows ◽  
Ian Stockford ◽  
Qingping He
Keyword(s):  
Rasch Modelling ◽  
Comparative Judgement

scholarly journals Assessment by Comparative Judgement: An Application to Secondary Statistics and English in New Zealand

2020 ◽  
Vol 55 (1) ◽  
pp. 49-71
Author(s):  
Neil Marshall ◽  
Kirsten Shaw ◽  
Jodie Hunter ◽  
Ian Jones
Keyword(s):  
New Zealand ◽  
Reliability And Validity ◽  
National Assessment ◽  
Student Work ◽  
Comparative Judgement ◽  
Assessment Tasks ◽  
Potential Applications ◽  
Valid Assessment ◽  
High Level

Abstract There is growing interest in using comparative judgement to assess student work as an alternative to traditional marking. Comparative judgement requires no rubrics and is instead grounded in experts making pairwise judgements about the relative ‘quality’ of students’ work according to a high level criterion. The resulting decision data are fitted to a statistical model to produce a score for each student. Cited benefits of comparative judgement over traditional methods include increased reliability, validity and efficiency of assessment processes. We investigated whether such claims apply to summative statistics and English assessments in New Zealand. Experts comparatively judged students’ responses to two national assessment tasks, and the reliability and validity of the outcomes were explored using standard techniques. We present evidence that the comparative judgement process efficiently produced reliable and valid assessment outcomes. We consider the limitations of the study, and make suggestions for further research and potential applications.

scholarly journals An information system design theory for the comparative judgement of competences

2018 ◽  
Vol 27 (2) ◽  
pp. 248-261 ◽  
Author(s):  
Tanguy Coenen ◽  
Liesje Coertjens ◽  
Peter Vlerick ◽  
Marije Lesterhuis ◽  
Anneleen Viona Mortier ◽  
...  
Keyword(s):  
Information System ◽  
System Design ◽  
Design Theory ◽  
Information System Design ◽  
Comparative Judgement
Sign in / Sign up

Export Citation Format

Share Document