Record Details

Replication Data for: Bridging the Grade Gap: Reducing Assessment Bias in a Multi-Grader Class

Harvard Dataverse (Africa Rice Center, Bioversity International, CCAFS, CIAT, IFPRI, IRRI and WorldFish)

View Archive Info
 
 
Field Value
 
Title Replication Data for: Bridging the Grade Gap: Reducing Assessment Bias in a Multi-Grader Class
 
Identifier https://doi.org/10.7910/DVN/BIORH8
 
Creator Kates, Sean
Paulsen, Tine
Yntiso, Sidak
Tucker, Joshua A.
 
Publisher Harvard Dataverse
 
Description Many large survey courses rely on multiple professors or teaching assistants to judge student responses to open-ended questions. Even following best practices, students with similar levels of conceptual understanding can receive widely varying assessments from different graders. We detail how this can occur and argue that it is an example of differential item functioning (or interpersonal incomparability), where graders interpret the same possible grading range differently. Using both actual assessment data from a large survey course in Comparative Politics and simulation methods, we show that the bias can be corrected by a small number of “bridging” observations across graders. We conclude by offering best practices for fair assessment in large survey courses.


These files should fully replicate the findings in "Bridging the Grade Gap: Reducing Assessment Bias in a Multi-Grader Class," accepted at Political Analysis in April, 2021.
 
Subject Social Sciences
Bayesian Aldrich-McKelvey scaling
 
Contributor Kates, Sean