Monitoring crew wellbeing through voice analysis

Two experiments in collaboration with Russian partners are using cosmonauts’ communication data to develop innovative technologies. This will enable their support crew on the ground to more effectively look after astronauts and cosmonauts’ emotional and psychological wellbeing when they’re in space.

There are also many environments on Earth where this technology could be useful – for example in the mining and maritime industries, where workers are often isolated and operating in dangerous or high pressure situations.

iVOICE: integrated VOIce analysis of satellite Communications Embedded in safety-critical environments

iVOICE is a cutting edge technology, developed with the Gagarin Cosmonaut Training Centre in Russia, which uses computers to predict when operators are getting tired, using automated analysis of their voices. The UK contribution is being led by researchers at the Centre for Space Medicine, part of UCL’s Mullard Space Science Laboratory in Surrey, and UCL’s Department of Speech, Hearing and Photonic Sciences.

Astronauts make perfect subjects for developing this technology, because they are very closely monitored – not just when they are on board the Space Station, but right through their training. This means very ‘clean’ data on their voice patterns can be collected, and this can easily be cross-checked against external factors (such as sleep patterns, diet and significant events in their activities).
Space does not just provide guinea pigs for this software: when deployed in remote places on the ground, reliable satellite networks are fundamental to making the system work.

It is planned to test iVOICE in open-cast mines in Peru, in cooperation with industrial partners. It ‘learns’ the usual speech patterns and voice quality of truck drivers and detects any unexpected levels of fatigue. These are reported to human controllers who can take appropriate action – for example instructing an operator to take a rest. This can save money for mining companies and, more importantly, the lives of drivers.
The system can be triggered by unexpected but subtle changes in pitch etc. in speech, which might not be apparent to listeners but can be detected by the computers’ sophisticated algorithms. One example of the power of this software is that it can identify whether a speaker has slept in the past 24 hours with an accuracy of 90%.

EPSILON: Embedded Psychological Support Integrated for LONg duration missions
Phase 1: VULCAN project

This project, currently in its final stages of agreement, builds upon the iVOICE technology to deliver a system capable of tracking voice characteristics of crew members engaged in long-term missions to identify vocal signals of changes in wellbeing. It does this using a pioneering technique known as Longitudinal Voice Analysis.

There is a sequence to operations of the system: recording voices; storing and communicating voices; analysing their content in terms of language (i.e. what words are used); analysing their content in terms of ‘voice features’ (i.e. what their voice sounds like); assessing changes in these features over time; detecting any anomalies or changes

There are of course challenges to developing this technology too: for example, making in-flight speech recordings in a non-invasive and unobtrusive way that protects crew privacy is important – you do not want your crew to ‘close down’ and stop communicating or invade the privacy of the crew. Ensuring the quality of the recordings is a challenge, simple voice channels are not adequate to record the subtle changes.
Astronauts are often very driven and determined to ‘get the job done’ and this might lead them to not report to their ground crew when they are feeling tired or stressed. This new technology, based on UK expertise in psychology, linguistics and space exploration, offers a way to unobtrusively monitor how they are really feeling.