Detecting drivers’ cognitive distraction by analyzing facial expressions

A group of researchers at École polytechnique fédérale de Lausanne (EPFL) has developed a system to detect the cognitive distraction of drivers by analysing their facial expressions.

Driving requires the constant coordination of many body systems and the full attention of the driver. Cognitive distraction (subsidiary mental load) is an important factor that decreases attention and responsiveness, which may result in human error and accidents.

The LTS5 group of researchers, led by Prof. Jean-Philippe Thiran, have published a paper which presents a study of facial expressions of such mental diversion of attention.

“First, we introduce a multi-camera database of 46 people recorded while driving a simulator in two conditions, baseline and induced cognitive load using a secondary task,” said Prof. Thiran.

“Then, we present an automatic system to differentiate between the two conditions, where we use features extracted from facial Action Unit (AU) values and their cross-correlations in order to exploit recurring synchronization and causality patterns.

“Both the recording and detection system are suitable for integration in a vehicle and a real-world application, e.g. an early warning system. We show that when the system is trained individually on each subject we achieve a mean accuracy and F-score of ∼ 95%, and for the subject independent tests ∼ 68% accuracy and ∼ 66% F-score, with person-specific normalization to handle subject dependency. Based on the results, we discuss the universality of the facial expressions of such states and possible real-world uses of the system.”

The paper, authored by Dr Anil Yüce, Dr Hua Gao, Mr Gabriel Cuendet and Prof. Jean-Philippe Thiran, from the LTS5, has been recently publihed in the IEEE Transactions on Affective Computing.