Post by Mary Burgess with help from Jess Mitchell, Josie Gray, Tracy Roberts, Amanda Coolidge, Carolee Clyne, Susan Doner, and Mirjam van Hasselt. 

Because of the pandemic, the last several months have involved pivoting post-secondary courses in B.C. from face to face to online delivery. Part of that pivot has required new ways of administering exams. The use of proctoring software has emerged as a popular solution, and some B.C. institutions are currently in the process of evaluating different proctoring products such as Proctorio, ProctorU and Respondus, while others are relying on video conferencing as a way of monitoring students during exam writing. As much as it enables continuity of course delivery, using the software comes with its own set of problems and some B.C. educators and institutions are outright rejecting these tools in favour of teaching methods that do not require exams. The pandemic has shone a light on how we use assessment practices without questioning their validity or purpose. This article looks at some of the problems created by the use of proctoring software, as well as problems created more generally by the use of exams as an assessment method.

Accessibility

Most exams are what is known as time-based assessment. That means students have a specific amount of time, at a particular time, to perform the assessment. Within an assessment, there may also be time limits on how long a student gets before they have to respond to a question.

Time-based face to face assessment disadvantages many students. For example:

• Those who cannot sit comfortably for long periods of time as a result of physical disabilities or because of uncomfortable settings (no heat or AC; those who have acquired a temporary injury) 
• Those who have anxiety disorders that prevent concentration in timed settings; with the pandemic, anxiety is even more likely as students navigate many challenging situations in their personal and professional lives
• Those who have attention disorders that prevent focus, especially with the distractions of others around them, and those in settings that prevent focus, such as caring for an elderly parent or a small child during the pandemic
• Those with learning disabilities may not be able to adequately express their learning in written form
• Exam questions and answers may be written using cultural knowledge not all students have

Online Exams

Online exam proctoring, now becoming much more prevalent due to the pandemic, presents some additional challenges, including but not limited to:

• Facial detection software built into many of these systems is racist
• Incoming and outgoing data on student computers is being surveilled by the software
• There is a financial burden to students who have to purchase a webcam for the purpose of exams
• Not all students have computers that are robust enough to install and run the software
• Not all students have their own computer in a private room
• Not all students have constant, reliable internet access (due to their geographic location or competing priorities in their family group, e.g., students doing home schooling while others work from home, etc.), and upgrading creates a financial burden

In addition to the above, the issue of cheating has also come to the fore with the use of online exams. Academic integrity is being discussed more than ever. Institutions, departments, and individual faculty are using a variety of methods to stop cheating both before it happens, with things like academic integrity pledges, and after it happens, with plagiarism detection software and the like.

Students have their own thoughts about exams being administered this way. A quick tour through institutional subreddits on the Reddit platform tells the story of students who fear for their privacy, are confused about what their rights are, are very aware of the power dynamic between themselves and their instructors, and, above all, want to succeed in their courses. Some students claim to have cheated the systems put in place to prevent cheating, and we hear similar stories from other sources, including the software companies themselves. The bottom line is that it is not a pretty picture.

Exams are not only problematic from an accessibility and inclusion lens. Teachers need to know whether their students learned what they intended them to learn. Unfortunately, exams instead often assess memory and the ability to regurgitate information without actually having learned it deeply in a way that enables application at a later date. In addition, the nature of exam season means that students will likely have several exams over a short period of time, thus rendering their performance a matter of stamina rather than a reflection of what they have actually learned.

Are they demonstrating learning?

Assessment of learning is vitally important to a student’s journey. In order to achieve mastery of course competencies, they need regular and actionable feedback. Assessments need to align with learning goals that are appropriate for the course level and the mastery expected. Bloom’s taxonomy is one of several tools and methods used to talk about levels of learning and mastery and to therefore design learning experiences and assessments. Educators need to think about what students should be able to do as a result of their learning. In some cases, a requirement may be the ability to analyze information using the learning, which requires much more mastery than simply remembering concepts such as formulas. David Jaffee points out that in having students study for exams, we are focussing their attention on a very short term goal rather than on the larger context in which the learning exists and will ultimately be applied.

Unfortunately, many exams only test remembering, and they end up testing a whole bunch of other things that have nothing to do with mastery or the ability to use the learning in a variety of ways. Exams that are held in auditoriums with several hundred students at the same time test a student’s ability to focus on a particular task when there are many other distractions around them. For some students, this is relatively easily done. For others, the inability to focus might be the result of anxiety, and that anxiety may prevent the student from recalling information they have studied diligently.

The notion of scaling education is almost always in direct opposition to what we know about how learning works. Classes with hundreds of students per educator create a situation where few thrive and the likelihood of many not having their learning needs met is high. Thinking of students or any one “group” of students, such as “those with disabilities”, as a monolith ignores the many ways in which needs, skills, and background affect learning.

Trauma-informed education

Dr. Stephen Porges’ research into the pathways that connect the brain to the body tells us that certain physiological states reduce our ability, during times of stress, to access information we have already learned. We also now know much more about trauma and its impact on our ability to learn new information. There are many examples of students for whom trauma may be a barrier to learning and to expressing their learning. A student whose grandparents went to residential school, for example, may be impacted by intergenerational trauma and could find that an exam setting provokes feelings that are barriers to focus, expression of learning, and feeling safe.

Given this information, we are in a position to better design learning and assessments to give students the very best chance at success. So, what do we do instead?

Use Formative AND Summative Assessment

Assessment can be formative, i.e. it happens during the learning process, or summative, i.e. it comes at the end of the learning process. For students to gain mastery, they need multiple attempts at learning with feedback along the way to guide them. Many final exams are worth a very large percentage of a student’s final grade in a course. It is not uncommon to see courses with two midterms worth 25% each and a final worth 50%. Students need formative assessment to help them identify where they need to do more learning. Educators need formative assessment to know whether their teaching techniques are working. Supporting students through their learning journey using formative assessment can help put much less emphasis on the necessity of a final exam and can spread the work of assessment over a term instead of piling it up at the end. One alternative strategy is to provide students with multiple opportunities throughout the course to show competency, using a variety of techniques including short, low risk, in-class quizzes and peer assessment. Students often only study for a final exam in the days leading up to it; ensuring there are regular opportunities for assessment throughout the course will require consistent acquisition of knowledge. Assessing learning along the way also gives students practice with knowledge retrieval processes. Recent research indicates that this practice strengthens learning and, in fact, enables higher order learning to take place.

Does it need to be time-based? Does it need to be written?

Exams have traditionally been administered over several hours at once. We have done it this way for a long time, but that does not mean it works. Time-based exams are a scheduling nightmare at institutions, and in most cases, this way of working is not an accurate reflection of how students will use the learning in real life situations. In most fields, nobody is going to ask you to do something in two hours, by yourself, with no access to any resources. A take-home exam can be a good alternative. Not only does it reduce the likelihood that your exam and its physical setting are not accessible to some students, it can also be an opportunity for students to reflect more deeply on their learning. Many instructors will be concerned about cheating with take home exams. One way to prevent that is to design questions that require students to use higher order thinking skills like analyzing and creating, using reference to learning from the course and other sources.

The ways in which we expect students to express learning could also be much more flexible than it currently is. Again, this is a case in which educators can look at what students need to learn in a course and make a decision about whether the method of expression of learning is germane to the assessment. Perhaps making a video or doing a presentation is more appropriate for a student moving into some fields. Perhaps a conversation is a better way to find out what a student knows. Some students who are extremely strong academically have major challenges when it comes to writing. We are currently penalizing those students who excel at oral communication over written. This is another example of where scaling education and tradition lets us down.

Co-creation with students

How about having the students design the exam questions? Co-creation of curriculum with students is an emerging way of teaching that is gaining ground due to its effectiveness. What if students each had to create four exam questions that reflected their knowledge of the course topic, and then answer each other’s questions? Creating questions requires deep thinking and makes students accountable for their own learning. In order to create multiple choice questions with plausible wrong answers, students must first master a topic. Question creation could happen throughout the term or only in the weeks leading up the exam. This method can also be used during a take-home exam in which students use their own learning and experience to come up with questions rather than answers to questions.

Applied learning

If what you are really interested in is enabling your students to apply what they learn in your course in different contexts, multiple choice exam questions are not going to get you there. Using case studies from real businesses, for example, can help assess whether students remember the math formulas you taught them as well as their ability to choose the right formula to solve a problem. Applied learning can also be much more engaging for students, motivating them to explore topics on their own and take control of their own learning. Like other methods, this way of engaging students can happen over an entire semester, or be used as a way of assessing learning at the end of the course in a final project or take-home exam that asks students to apply learning from across the course to a set of real world problems.

Student Reflection

At the University of Saskatchewan, Professor Hayley Hesseln in the College of Agriculture and Bioresources has been successfully using reflection as a way for students to demonstrate learning. She provides students with a rubric to guide their thinking, and as a summative way of assessing asks them to write about what they learned in the course, including how they will use their learning in the future. Using a rubric can simplify the assessment process for both students and educators because it focuses on specific and measurable goals, makes students aware of expectations, and helps connect them to their learning in ways that enable them to assess their progress. Jess Mitchell, in her UX course at Wilfred Laurier University, uses this method on a more frequent basis. At the end of each weekly class, she asks students to connect their new learning with how they see the world and what they notice they are thinking about. She also encourages them to think about their learning and what supports they need. These prompts enable students to explore more deeply topics they are passionate about and to take agency over their own learning.

What will you do now?

Exams have been a regular part of the post-secondary experience for a very long time, and no doubt they will continue to be used for a long time to come. We know more than we used to about how learning happens and about what can prevent students from actually learning and expressing learning. Do you have an exam in your course? What might it look like to do something different? What supports would you need? If you are ready for a change, seek out your institution’s learning and teaching experts, who are always really excited about challenges like this! Talk to colleagues who might want to take the journey with you. Doing things in community can feel a lot less risky. If you can change even a couple of small ways in which you assess students’ learning, you will have made progress. You could even involve your students in the decision making about how to assess their learning.

At BCcampus we are collaborating with educators who are shifting to assessment methods that have a positive impact on student learning. If that sounds like you, please connect with us so we can talk about how we can support you!