Use of online therapy session data to develop behavioural markers for cognitive outcomes in non-pharmacological intervention

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

View graph of relations

Author(s)

  • Yong Li
  • Janet H Hsiao
  • Jacky Chak Pui Choy
  • Maggie Ma
  • Gloria HY Wong

Detail(s)

Original languageEnglish
Article numbere082834
Journal / PublicationAlzheimer's and Dementia
Volume19
Issue numberS24
Online published25 Dec 2023
Publication statusPublished - Dec 2023

Abstract

Background:  Evidence-based non-pharmacological interventions such as group-based cognitive stimulation therapy (CST) involve complex interpersonal interactions based on the putative mechanism of person-centred engagement. Empirical evidence of the hypothesised mechanism of action is lacking. Behavioural markers such as eye movement patterns can be used to predict cognitive ability. The rapid development of online interventions during the pandemic has made behavioural information such as eye movement patterns during therapy more readily available for hypothesis testing on mechanisms.

Method:  We retrieve behavioural data using random time sampling of virtual CST session from people with mild to moderate dementia. Intervention characteristics of interest are frequency and relevance of self-identity stimuli presented during the therapy session. Behavioural markers of interest are visual attention, language output, and facial expression. For visual attention, we compare subject-independent and subject-dependent protocols of automatic gaze estimation, using private and public (MPIIFaceGaze) dataset for pre-training.

Results:  Person-centredness of the intervention can be operationalised based on a pre-intervention assessment of self-identity roles and video annotation. For determination of eye movement pattern, pretraining findings using 45,000 public train images, 6,420 private train images, and 600 private test images showed an average angle error of 9.79 using subject-independent protocol, versus 5.91 using subject-dependent protocol.

Conclusion:  Behaviours captured on computer cameras during online therapy sessions can be automatically estimated reliably, especially with subject-dependent protocol for eye movement and gaze analyses. This is a proof of concept for further development of behavioural markers, with advantages of simultaneously observing multiple interacting participants without Hawthrone effect and distraction in the group therapy. Further protocol finetuning using a larger data set, and models incorporating hand-coded markers of person-centred engagement for outcome prediction will be the next steps.

Bibliographic Note

Research Unit(s) information for this publication is provided by the author(s) concerned.

Citation Format(s)