Real-time motion capture algorithm for smart wearables

Cerebral Palsy (CP) is a physical disability that affects the movement and posture of a person. It is the most common physical disability that occurs in children. SMARTCLAP is a multidisciplinary research project whose main aim is to develop a revolutionary device that aids occupational therapy for children with CP. SMARTCLAP places the user at the centre of the design process and is motivated by the child’s functional goals set during occupational therapy. By means of this design approach, SMARTCLAP aims to increase the motivation of a child during therapy sessions, in which the parents or the guardians may also be involved. This contributes towards developing positive behaviour and improving the social interaction of the child.

In particular, the SMARTCLAP project is in the process of developing a novel, hand Motion

Capture Algorithm (MCA) combined with a smart, wearable glove to be used in the rehabilitation of children, where the child can use it to play a serious, Augmented Reality (AR)based game. The system aims to offer a cheap, user-centred design option, which is tailormade for a particular child and makes use of current technologies and 3D printing.

Hand and finger movements are captured by several Inertial

Measurement Units (IMUs) attached to the device, which are positioned in the middle of the proximal and intermediate phalange of each finger and on the back of the palm. Positioning data from each sensor is captured by an ATmega328 microcontroller and sent via Bluetooth to a tablet or a smartphone which runs an AR game.

The MCA generates a model of the hand and its movements using the kinematics defined by the International Society of Biomechanics, which are interfaced with the AR system. The AR game then sends back visual and auditory feedback to the user.

The movements and positioning of the different sensors with respect to each other are achieved using the mathematical theory of dual-Quaternions. Dual-Quaternions offer a highly computationally efficient and unique way of representing rotations and translations in a three-dimensional space.

The SMARTCLAP project is funded by the Malta Council for Science and Technology (MCST) (project reference R&I2019-003-T).

The project is led by Philip Farrugia from the Department of Industrial and Manufacturing Engineering at the University of Malta.

Other members of the research team are Owen Casha, Simon G. Fabri, Mario Farrugia, Matthew Bonello, Alexiei Dingli, Nathalie Buhagiar and David Sciberras. Further project details may be reviewed at: https://www.um.edu.mt/ projects/smartclap/.

The first prototype of the wearable device being tested on three fingers. Data is gathered from the sensors on the fingers and palm, captured by the microcontroller encased on the forearm and sent via Bluetooth to the game.