Facial expression is among the primary non-verbal means in human communication – for example, a smile can express happiness or positive opinions.

However, existing VR and AR head mounted displays (HMDs) such as HTC Vive and MS Hololens occlude the wearer’s face significantly, especially the eye region which is one of the most emotion-salient facial parts, which severely prohibits natural interactions between the user and the virtual environment.

To alleviate this problem, this project integrated advanced biometric sensors into the HMD in an unobtrusive manner to capture fine-scale dynamic facial movements which are further mapped to a user-specific 3D face model to recover the user’s facial expression with high-fidelity.

Read the project publication, Realistic Facial Expression Reconstruction for VR HMD Users.

Parkinson's disease (PD) affects 30 million people worldwide and is the second commonest neurodegenerative disorder – symptoms include tremors, muscle rigidity, dropping posture, walking difficulty, loss of facial expressivity, loss of vocal projection.

Experts in the field state that biomarkers (which include clinical, biochemical, genetic, proteomic, or neuroimaging) may help early detection, differentiation of atypical Parkinson's and earlier symptom control.

Clinical biomarkers, such as motor function are particularly attractive as they enable early monitoring, are non-invasive and may be applied to a number of clinical conditions affecting a similar demographic. In addition to aiding at-home monitoring and assessment, several studies have also shown that gait performance in PD can be improved by applying continuous external rhythmic auditory or visual cues.

This project aims to develop a novel multi-sensory wearable device, incorporating multiple sensors, to detect physical activity, gait, posture, voice and facial EMG.