Before the COVID-19 pandemic, educators commonly assessed student engagement using student self-report measures, teacher reports, and observational measures. These measures work for in-person learning environments where student participation and connectivity can be easily visible.
Now most teaching and learning occurs in virtual spaces, where instruction is delivered through learning management systems (e.g., Google Classroom) either synchronously or asynchronously. Teachers and school leaders have to engage students — and evaluate their engagement — in a remote environment with little formal or comprehensive training and often a limited ability to even see their students.
There has been a great deal of writing recently about improving student engagement, but few resources provide guidance around measuring student engagement during remote learning.
Based on a close read of the existing research and resources, and our own in-house expertise, Bellwether’s evaluation team is currently helping three clients to design and implement tools to evaluate virtual student engagement. While the measures themselves have not changed much since the pandemic, the uses and evaluation of these measures must adapt to our current realities, like many other approaches in education, to better serve and engage students and families.
Here are two key considerations for educators and school leaders to keep in mind when evaluating virtual student engagement:
There is no one-size-fits-all model for measurement
Despite the desire for generalizable measures that can be used to capture engagement patterns across learning environments and schools, student engagement measurement tools must be adapted to classroom designs. In particular, metrics that provide incredibly useful data in synchronous remote learning environments (e.g., hourly participation in remote class sessions and quality of engagement in remote class discussions) become obsolete in asynchronous models where more absolute measures reign (e.g., frequency or quantity of online learning activities completed).
Pervasive educational inequities, such as lack of widespread, high-speed internet access, mean that evaluators should not rely on strictly absolute measures of engagement. Other factors such as technical/connectivity issues, comfort with learning management systems, or clarity of instructions and classroom organization may inhibit students’ ability to engage.
Incorporating innovative metrics, such as participation in wellness checks and responses on student and family engagement surveys, can help educators develop a clearer picture of student engagement patterns. These measurement tools can also shape plans for, and adjustments to, pacing charts and instructional plans in order to ensure students’ needs are being met. Student and family engagement surveys, which are commonly distributed online, can also help to inform more trauma-sensitive and culturally competent educational planning and practice. Specifically, engagement surveys that collect contextual information on topics such as how families prefer to be contacted and interact with schools, how families are coping with the collective trauma of the pandemic, and how families are navigating the national racial unrest in response to the recent killings of Black Americans can help educators make future lessons more relevant to student and family needs.
Student engagement does not occur in a vacuum
Because student engagement is influenced by students’ relationships and interactions with peers and family members as well as their educators, measurement tools that incorporate students’ levels of connectivity across the board rather than individual-level behaviors and performance alone may be best. For example, standards- and project-based learning models can promote student connection with peers, encourage one-on-one meetings with teachers, and incorporate family members to move beyond traditional measures of student engagement with materials and their direct instructors. These kinds of assignments can also help educators move from a model of measuring assignment completion to measuring quality of student engagement and interactions with their peers.
Finally, research indicates that a variety of communication methods can influence students’ perceptions of and relationships with their educators, not just face-to-face (or synchronous) interactions. In this way, written communications (e.g., emails), videos, Zoom chats, and other methods of fostering engagement can be used to evaluate student engagement as well.
We’ve been working closely with researchers and schools to reimagine data collection throughout the COVID-19 crisis, and recently hosted a webinar about educational evaluation in light of the pandemic. Reach out to our evaluation team if you’d like support from our team.