Next level proctored exam for proficiency testing in Primary Care Education: an observatory study on efficiency and accuracy and on exam outcome. (Preprint)
BACKGROUND The COVID19 pandemic affected education and assessment and led to a complex planning. Therefore, we organised the proficiency test for admission to Family Medicine as a proctored exam. To prevent from fraud we developed a virtual supervisor app tracking and tracing candidates’ behaviour. OBJECTIVE To assess efficiency and accuracy of the proctored exam procedure and to test the impact on the exam scores. METHODS The app operates on three levels to register events: recording of actions, analyses of behaviour and live supervision. Each suspicious event is given a score. To assess efficiency we inventoried the technical issues and the interventions. To test accuracy we counted the number of suspicious students and behaviours. To test the impact of the supervising app on students’ exam outcome we compared the scores between the proctored and the on campus group. Candidates were free to register for off or on campus participation. RESULTS 593 candidates subscribed to the exam: 472 (79%) candidates used the supervisor app and 121 (20%) were on campus. Test results of both groups were comparable. We registered 15 technical issues in off campus context. Two candidates experienced a negative impact on the exam due to the technical issue. The app detected 22 candidates with a suspicious level >1, mainly increased due to background noise. All events occurred without fraud purpose. CONCLUSIONS This pilot study demonstrated that a supervisor app with recording and registration behaviour is able to detect suspicious events without an impact on the exam. Background noise was the most critical event. There was no fraud detected. A supervisor app registering and recording behaviour to prevent from fraud during exams is efficient and not affecting the exam outcome. In future research, a controlled design should compare the cost-benefit balance between the complex intervention of the supervisor app and the combination of the candidates’ awareness of being monitored with a safe exam browsing plug in. CLINICALTRIAL Not applicable