Simulating One-on-One Teaching in Large Online Lectures with the Thousand Faces Avatar

Project: Research

View graph of relations


Online lectures on platforms like Zoom are widely used all around the world since the COVID-19 pandemic. Though a few studies showed that the teaching quality is relatively good, empirical observation has shown that students get distracted much easier than during face-to-face lectures - and a fortiori compared to a one-on-one teaching session. Previous studies indicate that virtual online environments (including virtual spaces and the use of avatars) can partially compensate for the sense of social presence lost in video conference online lectures. Leveraging on these observations as well as personal experience gathered in the past two years, we propose an original and scalable strategy readily applicable to video-online teaching scenarios (and beyond), bringing back some of the advantages found in a one-on-one personal teaching experience, even when the teacher is imparting a lesson to a large, non-collocated class.More concretely, we are applying for funding to support the development of a bidirectional semi-autonomous avatar simulating a one-on-one teaching and learning experience. The use of avatars and virtual environments provides the equivalent of a classroom: a neutral and abstract space dedicated to the teaching and learning experience, separated from the private sphere. For these reasons, it has been gaining momentum in online teaching platforms , and hailed as capable of eliminating distraction, promoting engagement, and encouraging freedom of expression, among other positive traits. Their principle is simple: each participant (including the teacher) is provided with one's own (tunable) avatar in a virtualized classroom. While this strategy simulates the dedicated space of a classroom, it does so in detriment of the capacity of personal assessment from the side of the teacher. The problem is exacerbated by the fact that online teaching and learning platforms allow for an arbitrary large number of students.Our proposal is geared toward solving this problem without restricting the number of students, since this is one of the advantages of the technology. The proposed system extends the use of avatars as follows. As with other systems, students see a single teacher avatar, through which the main teacher imparts the lesson. However, when appropriate, control can be passed to a teaching assistant to attend to a single student request, while the others continue listening to the main avatar teacher. Similarly, the teacher(s) will see a unique student avatar representing the whole cohort of students. This single student avatar splits automatically as a student "raise a hand" online to ask a question. This gives the choice to the teacher to pass control of the avatar as seen by that student only to a teacher assistant (or other teachers, see below), or to address the request in front of everybody. Our hypothesis is that since the illusion of a one-on-one teaching situation is both experienced by the teacher and the student, the experience will lead to a more personal and natural form of interaction, something greatly missed in the present situation. We need to work out the conditions for this to be true. Indeed, how the illusion is crafted will profoundly affect the experience. In the same way a small physical or behavioral imperfection in an otherwise realistic anthropomorphic robot can make their appearance "uncanny", a good avatar that sometimes split or react in unnatural ways while handling questions may create an "uncanny valley" as we work towards a "perfect" but still mediated form of inter-personal communication. The research is important and timely as online-mediated communication is here to stay. Apart from such operational questions, our work may also lead to fundamental insights at a more fundamental level regarding the transformative power of online technologies, such as to which extent an avatar can be considered an interface to present ourselves, or a new way of being (as hinted by the extreme use by millennials who would refuse to trade the technology for a meeting in person).We will strive to deploy the technology on top of an online teaching platform like Zoom (with the help of CityU IT department) so as to take advantage of an already familiar infrastructure. The virtual space and the avatars can be displayed on the computer monitor; however, this research paves the way to the use of more advanced AR technology to insert 3D avatars (in particular that of the teacher) in the real physical space of the student (home, public space), either using a tablet or a head mounted display. In time, the system can be enhanced by introducing semi-automatic behaviors with the help of a context aware AI, enhancing the realism of the one-on-one simulation (choice of gestures and posture, but also character, voice, gender, etc.).Interestingly, as students and teacher(s) can be anywhere in the world, this strategy may be applied to create a one-on-one personalized massive open online course (MOOC), something that sounds impossible. The key is to understand than in our setup, one can not only recruit a large number of students, but also a (smaller) group of teachers working in concert and puppeteering (and splitting) a unique avatar when required. In the same vein, one can conceive a physical representation of the (single) class avatar, for instance a commercial telepresence robot that could be transported to special locations, such as museums or other locations (including overseas) inaccessible to many. This concept, proposed by the PI when working for the Uruguayan government under the PLAN CEIBAL  initiative, was underway before he moved to Hong Kong - the telepresence robot was conceived as a school bus that could be sent to the country's capital or another country for that matter, and attended and "walked by" a local teacher or guide.


Project number6000778
Grant typeTDG(CityU)
Effective start/end date1/01/22 → …