Brooke Rhees’ MS Thesis Defense
Title: A Semi-Automatic Grading Experience for Digital Ink Quizzes
Friday, January 13th at 1:30 pm
In 3350 TMCB
Advisor: Dan Olsen
Teachers who want to assess student learning and provide quality feedback are faced with a challenge when trying to grade assignments quickly. There is currently no system which will provide both a fast-to-grade quiz and a rich testing experience. Previous attempts to speed up grading time include NLP-based text analysis to automate grading and scanning in documents for manual grading with recyclable feedback. However, automated NLP systems all focus solely on text-based problems, and manual grading is still linear in the number of students. Machine learning algorithms exist which can interactively train a computer quickly classify digital ink strokes. We used stroke recognition and interactive machine learning concepts to build a grading interface for digital ink quizzes, to allow non-text open-ended questions that can then be semi-automatically graded. We tested this system on a Computer Science class with 361 students using a set of quiz questions which their teacher provided, evaluated its effectiveness, and determined some of its limitations. Adaptations to the interface and the training process as well as further work to resolve intrinsic stroke perversity are required to make this a truly effective system. However, using the system we were able to reduce grading time by as much as 10x for open-ended responses.