Augmenting Access to Embodied Knowledge Archives: A Computational Framework

With the burgeoning use of digital technologies in safeguarding intangible and living heritage, memory institutions have produced a significant body of material yet to be made accessible for public transmission. The quest for new ways to unlock these massive collections has intensified, especially in the case of embodied knowledge embedded in complex formats, such as audiovisual and motion capture data.

This study examines a computational workflow that combines posture recognition and movement computing to bridge the gap in accessing digital archives that capture living knowledge and embodied experiences. By reflecting on how embodied knowledge is sustained and potentially valorised through human interaction, we devise a series of methods utilising vision-based feature extraction, pose estimation, movement analysis, and machine learning. The goal is to augment the archival experience with new modes of exploration, representation, and embodiment.

This article reports the computational procedures and algorithmic tools inspected through two use cases. In the first example, we visualise the archives of the Prix de Lausanne, a collection of 50 years of video recordings of dance performances, for archival exploration through the dancers' poses. In another experiment, movement encoding is employed to allow multimodal data search via embodied cues in the Hong Kong Martial Arts Living Archive, a comprehensive documentation of the living heritage of martial arts that is chiefly comprised of motion-captured performances by masters.

Though holding different application purposes, both projects operate on the proposed framework and extract archive-specific features to create a meaningful representation of human bodies, which reveals the versatile applications that computational capacities can achieve for embodied knowledge archives. The practices also represent a model of interdisciplinary involvement where the archivists, computists, artists, and knowledge holders join hands to renew strategies for archival exploration and heritage interpretation in a new light.

© 2024, Alliance of Digital Humanities Organisations. All rights reserved.