Early Fusion of Visual Representations of Skeletal Data for Human Activity Recognition

Abstract

In this work we present an approach for human activity recognition which is based on skeletal motion, i.e., the motion of skeletal joints in the 3D space. More specifically, we propose the use of 4 well-known image transformations (i.e., DFT, FFT, DCT, DST) on images that are created based on the skeletal motion. This way, we create "activity"images which are then used to train four deep convolutional neural networks. These networks are then used for feature extraction. The extracted features are fused, scaled and upon a dimensionality reduction step they are given as input to a support vector machine for classification. We evaluate our approach using two well-known, publicly available, challenging datasets and we demonstrate the superiority of the fusion approach. © 2022 ACM.