By Colin Madland, PhD candidate and sessional instructor, University of Victoria

Assessment is a contentious and high-stakes topic in higher education, and concerns have escalated with the emergence of the COVID-19 pandemic and generative artificial intelligence (AI) models. In the fall of 2021, I embarked on a BCcampus Student Research Fellowship to examine technology-integrated assessment practices in higher education.

To begin, we conducted a literature review to establish how higher education instructors integrate technology and assessment (Madland et al., 2024b). We purposely explored papers written by instructors in a wide variety of disciplines who are compelled to use technology when they assess learning. We excluded instructors who study assessment as a disciplinary area. We looked at these articles through the lens of the assessment design in a digital world framework published by Bearman et al. (2022). While there were areas of overlap between the literature and the framework, primarily the importance of understanding the different purposes of assessment, there were also themes that emerged in the literature not modeled in the framework (e.g., instructor workload, academic integrity), and components of the framework not evident in the literature (e.g., digital literacies). These incongruities suggested a need to revise the model, which led to envisioning what a technology-integrated assessment model might look like.

The resulting technology-integrated assessment framework (Madland et al., 2024a) serves as a starting point to understand how to improve technology-integrated assessment practices in higher education in British Columbia and beyond. The technology-integrated assessment framework consists of four components for instructors to consider when planning assessment.

1. The purposes of assessment
   • Assessment of learning
   • Assessment for learning
   • Assessment as learning
     
2. Duty of care
   • Bias
   • Inclusion
   • Relationships
   • Ethical EdTech
     
3. Technology acceptance
   • Performance expectancy
   • Effort expectancy
   • Social influences
   • Facilitating conditions
     
4. Assessment design
   • Measurement theory
   • Academic integrity
   • Relevance
   • Reciprocity

We considered the framework in light of the Five Rs of Indigenous education: respect, relevance, reciprocity, responsibility, and relationships (Kirkness & Barnhardt, 1991; Tessaro et al., 2018). Respect and responsibility are embedded in the duty of care component while relationships, reciprocity, and relevance are explicit in the framework. As we continue our research, we will look at the relationships between the factors and sub-factors in the framework.

I am grateful to my supervisor, Dr. Valerie Irvine, for her keen insight and expert contributions, as well as to my committee members, Dr. Chris DeLuca (Queen’s University) and Dr. Okan Bulut (University of Alberta), whose diverse wisdom around assessment are invaluable.

I am also appreciative of BCcampus, especially Leva Lee, Britt Dzioba, Rebecca Shortt, and Gwen Nguyen for their support through the BCcampus Student Fellowship program. The program provides B.C. post-secondary educators and students funding to conduct research on teaching and learning, as well as explore evidence-based teaching practices focusing on student success and learning.

References

Bearman, M., Nieminen, J., & Ajjawi, R. (2022). Designing assessment in a digital world: An organising framework. Assessment & Evaluation in Higher Education, 48(3).

Kirkness, V. J., & Barnhardt, R. (1991). First Nations and Higher Education: The Four R’s– Respect, Relevance, Reciprocity, Responsibility. Journal of American Indian Education, 30(3), 1–15.

Madland, C., Irvine, V., DeLuca, C., & Bulut, O. (2024a). Developing the Technology-Integrated Assessment Framework. The Open/Technology in Education, Society, and Scholarship Association Journal, 4(1), 1–19.

Madland, C., Irvine, V., DeLuca, C., & Bulut, O. (2024b). Technology-Integrated Assessment: A Literature Review. The Open/Technology in Education, Society, and Scholarship Association Journal, 4(1), 1–48.

Tessaro, D., Restoule, J.-P., Gaviria, P., Flessa, J., Lindeman, C., & Scully-Stewart, C. (2018). The Five R’s for Indigenizing Online Learning: A Case Study of the First Nations Schools’ Principals Course. Canadian Journal of Native Education, 40(1), 125–143.