One major disruption caused by COVID-19 has been the shift from in-person to online learning. However, a benefit of this change has been the expansion of online educational technologies. Instructors and students have gained firsthand experience with the online learning tools, and colleges and universities have tested if the technology can scale.

This insight into online courses has given students and instructors a sense of whether the technology meets the needs of educational environments. It has even inspired some students to take on the challenge of improving aspects of the technology they found lacking.

Nonverbal Communication in Music Education

Although many aspects of teaching translate well to an online environment, not all subjects are equal. Music, for example, involves a lot of nonverbal feedback from an instructor while students play their instruments. A University of California San Diego (UCSD) music and computer science professor, Shlomo Dubnov, identified this as an issue.

“In a music classroom, non-verbal communication such as facial affect and body gestures is critical to keep students on task, coordinate musical flow and communicate improvisational ideas,” said Dubnov. “Unfortunately, this non-verbal aspect of teaching and learning is dramatically hampered in the virtual classroom where you don’t inhabit the same physical space.”

To address this challenge, Dubnov enlisted the assistance of Ross Greer, a Ph.D. student at UCSD. Their solution involved creating an eye-tracking device that could give students information about an instructor’s gaze. Gaze is important in music. For example, a conductor uses eye contact with members of an orchestra to single out a member and provide instruction. Similarly, gaze is an essential aspect of music education. Yet, in an online environment, it is difficult for students to receive this direct feedback.

Their solution was to integrate eye-tracking software into the learning environment. The device tracks which student an instructor is looking at on their monitor and notifies the student that the instructor is looking at them. This allows the instructor to communicate with the student in this nonverbal way.

“We divided the screen into 91 squares, and after determining the location of the teacher’s face and eyes, we came up with a ‘gaze-estimation’ algorithm [sic] that provides the best estimate of which box—and therefore which student—the teacher is looking at,” explained Greer.

So far, the team has conducted a pilot study to test the potential of this feature. Seventy-two students were recruited to test the system. Of those participating, 87 percent believed that they felt they could better establish a communicative relationship with the conductor. However, there is still work to be done. If an instructor is too far from the screen, it is hard to read their eye movements. But this issue could be addressed with higher-quality camera resolution and more training data. This is a promising area for further research.

Addressing the Isolation of Online Education

Another Ph.D. student at UCSD, Matin Yarmand, created a research project to evaluate the experience of students and instructors teaching online over this past year. Although there were many anecdotal reports about people’s experiences, it was important to ground these stories in actual data. The goal of the research was to discover how to better support instructors and students.

Yarmand teamed up with Jaemarie Solyst, a Ph.D. student at Carnegie Mellon University, and Nardir Weible, an associate professor at UCSD. Their research identified that one barrier for instructors was the students’ lack of social presence. Many students did not turn on their cameras in the classroom, which made it difficult for the instructor to tell if students were engaged.

Students resisted turning on their cameras for several different reasons. Some felt awkward on camera, others were eating or doing other activities during class that they did not want others to see, and some students just refused to use a camera. Many students did not understand that the cameras were aiding the instructors during lessons.

This does not mean that the students were inactive during class. Students were drawn toward chat features. One respondent explained, “I’m more comfortable asking questions in chat, since I feel less like I’m interrupting the lecture.”

Yarmand and his team identified several potential ways to give instructors feedback in the absence of cameras. Technology that reads facial expressions could give instructors feedback about students’ affect without the need for students to be visible on screen. Video feeds could also hide the images of the students from other students. This would mimic the in-classroom experience of having all eyes on the instructor.

More research is needed, but the authors are confident that this research will continue past the disruption of COVID-19. “Some classes will be back in person,” Weibel said. “Some will be only online and some will be hybrid, but I think online learning is probably here to stay.”