Use of online therapy session data to develop behavioural markers for cognitive outcomes in non-pharmacological intervention
Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
Original language | English |
---|---|
Article number | e082834 |
Journal / Publication | Alzheimer's and Dementia |
Volume | 19 |
Issue number | S24 |
Online published | 25 Dec 2023 |
Publication status | Published - Dec 2023 |
Link(s)
DOI | DOI |
---|---|
Permanent Link | https://scholars.cityu.edu.hk/en/publications/publication(cc692332-c3bf-41c9-b520-f0f2e8eb1339).html |
Abstract
Background: Evidence-based non-pharmacological interventions such as group-based cognitive stimulation therapy (CST) involve complex interpersonal interactions based on the putative mechanism of person-centred engagement. Empirical evidence of the hypothesised mechanism of action is lacking. Behavioural markers such as eye movement patterns can be used to predict cognitive ability. The rapid development of online interventions during the pandemic has made behavioural information such as eye movement patterns during therapy more readily available for hypothesis testing on mechanisms.
Method: We retrieve behavioural data using random time sampling of virtual CST session from people with mild to moderate dementia. Intervention characteristics of interest are frequency and relevance of self-identity stimuli presented during the therapy session. Behavioural markers of interest are visual attention, language output, and facial expression. For visual attention, we compare subject-independent and subject-dependent protocols of automatic gaze estimation, using private and public (MPIIFaceGaze) dataset for pre-training.
Results: Person-centredness of the intervention can be operationalised based on a pre-intervention assessment of self-identity roles and video annotation. For determination of eye movement pattern, pretraining findings using 45,000 public train images, 6,420 private train images, and 600 private test images showed an average angle error of 9.79 using subject-independent protocol, versus 5.91 using subject-dependent protocol.
Conclusion: Behaviours captured on computer cameras during online therapy sessions can be automatically estimated reliably, especially with subject-dependent protocol for eye movement and gaze analyses. This is a proof of concept for further development of behavioural markers, with advantages of simultaneously observing multiple interacting participants without Hawthrone effect and distraction in the group therapy. Further protocol finetuning using a larger data set, and models incorporating hand-coded markers of person-centred engagement for outcome prediction will be the next steps.
Method: We retrieve behavioural data using random time sampling of virtual CST session from people with mild to moderate dementia. Intervention characteristics of interest are frequency and relevance of self-identity stimuli presented during the therapy session. Behavioural markers of interest are visual attention, language output, and facial expression. For visual attention, we compare subject-independent and subject-dependent protocols of automatic gaze estimation, using private and public (MPIIFaceGaze) dataset for pre-training.
Results: Person-centredness of the intervention can be operationalised based on a pre-intervention assessment of self-identity roles and video annotation. For determination of eye movement pattern, pretraining findings using 45,000 public train images, 6,420 private train images, and 600 private test images showed an average angle error of 9.79 using subject-independent protocol, versus 5.91 using subject-dependent protocol.
Conclusion: Behaviours captured on computer cameras during online therapy sessions can be automatically estimated reliably, especially with subject-dependent protocol for eye movement and gaze analyses. This is a proof of concept for further development of behavioural markers, with advantages of simultaneously observing multiple interacting participants without Hawthrone effect and distraction in the group therapy. Further protocol finetuning using a larger data set, and models incorporating hand-coded markers of person-centred engagement for outcome prediction will be the next steps.
Bibliographic Note
Research Unit(s) information for this publication is provided by the author(s) concerned.
Citation Format(s)
Use of online therapy session data to develop behavioural markers for cognitive outcomes in non-pharmacological intervention. / Li, Yong; Chan, Antoni B.; Hsiao, Janet H et al.
In: Alzheimer's and Dementia, Vol. 19, No. S24, e082834, 12.2023.
In: Alzheimer's and Dementia, Vol. 19, No. S24, e082834, 12.2023.
Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review