Recently, IMC Professor Judy Franks presented an interesting issue: her student evaluation feedback indicated that her students absolutely loved her course, but were overwhelmed with the workload. In addition, she was asked to teach three sections of the course in the upcoming quarter. Combined, these issues pointed to the need to streamline the assessments in her course to decrease student time spent completing them as well as faculty time spent grading. Professor Franks wanted to use technology to maintain the rigor of her assessments and make it easier to provide enough feedback to each student.
We focused on two assessments in the course: Programmatic Buying, which we updated using Zaption, and The Great Schlep, which we updated using a Canvas Quiz in combination with draw.io, a Google tool.
Programmatic Buying
The past iteration of the Programmatic Buying assessment asked students to watch a 37 minute webinar, then answer questions in essay format. We used Zaption to edit the webinar and embed questions into the video, rather than having students answer them afterwards.
The webinar featured several Question and Answer sessions with the moderators. By eliminating these, we were able to cut the length of the webinar from 37 minutes to 20 minutes. We added thirteen questions throughout the video: four multiple choice and nine short answer. At the end of the video, students were asked to submit an essay response answering global questions about the video.
Professor Franks implemented this assessment in both on-ground and online courses; feedback was very different between the two groups of students. Professor Franks stated, “The online students LOVED it. And, their work product coming out of Zaption was fantastic. The full-time (on-ground) students didn’t like it (overwhelming majority). They said they would rather have the instructor lecture and facilitate discussion vs. working off a video. They felt it wasn’t nearly as engaging.” One student stated, “I liked the Zaption tool as it helped me hone in on the lesson, especially since the narration was a bit fast and confusing. Having the interactive questions helped me with comprehension.”
The Great Schlep
An example question from Prof. Franks’ The Great Schlep Assessment.
The past iteration of the Great Schlep assessment asked students to watch a short case study video, which demonstrated four major theories from the course, and answer a four part essay prompt. Professor Franks suggested that we create a Canvas Quiz so students could submit a combination of auto-graded (multiple choice and matching), short answer, and longer essay responses.
To decide which parts of the original essay prompt would be auto-graded, we went through the prompt and color-coded and sorted the prompt by Bloom’s Taxonomy levels: remember, understand, apply, analyze, evaluate, and create. We knew that we could use auto-graded questions for information students needed to remember and understand, and possibly for information students needed to apply, analyze, or evaluate.
One of the major parts of the Great Schlep case study involved understanding how the actions of different groups of people interacted with each other, so Professor Franks wanted to use graphic network maps to represent the interactions. We developed a series of ten questions based on network maps, and used draw.io, a Google drawing tool, to create the maps.
Canvas quiz statistics showed that the network map questions differentiated well between students who did well on the overall quiz. Professor Franks stated that “the [assessment] got to a much deeper level of comprehension. The scores were mixed because it demanded much more of the students.”
If you’re interested in updating assessments in your course using technology tools, please contact Instructional Technologist Jackie Wickham at jackie@northwestern.edu.