There were nearly 37,500 traffic accident fatalities in 2016 in the US., with distraction and fatigue responsible for 10% and 3% of avoidable fatalities. Two seconds of driver distraction is all it takes to double the probability of a collision.

Several car companies are attempting to tackle this issue, but their methods are proprietary and not available for the benefit of most drivers. The cheapest and most reliable way of detecting driver drowsiness or distraction is with a camera and image processing. Other solutions are either too expensive or unrealistic (think EEG electrodes or unreliable biosensors).

In the real world, a python program wouldn’t be very useful because drivers don’t drive with their laptops wide open, so we decided to create an iPhone App that would serve as a prototype for the real-world application of the trained model using the OpenCV framework, a precompiled Dlib library for the image processing operations and CoreML tools for converting the Keras model to a format deployable to iOS using CoreML.

Specifically, a colleague named Shail Shah and I:

• Trained a deep learning model to detect driver facial annotations in a small Spark cluster using Keras/Elephas with a dataset of 200,000 images

• Produced an algorithm to interpret facial annotations to estimate driver drowsiness or distraction with OpenCV. Based this off a subjective measure devised by Swedish psychologists called the Karolinska Sleepiness Scale going from “Extremely Alert” (1) to “Extremely Sleepy, can’t keep awake” (10)

• Developed an iPhone app to be used on a car’s dash and tested the algorithm on a driver video dataset.