Skills:
Python
Open CV
Collaborators:
Lisa Aoyama
Cam Brookhouse
Disharee Mathur
MPzzz, Don't Let a Snooze Kill your Voice
MPzzz is a project displaying how machine learning can be used in an unethical application. By applying libraries dlib, maintained by Davis King, OpenCV, originally created by Intel, face_recognition, created by Adam Geitgey, and Drowsiness_detection, committed by Akshay Bahadur, we were able to detect faces, identify Members of Parliaments, and then track their eyes to find out when they were sleeping. After they fell asleep we then recorded how long they slept for and Tweeted a photograph of them asleep along with the duration they were asleep. The intention was to alert the politician's constituents, causing unrest and democratic action.
(The Tweet above is fictional, and the statistics provided are purely for comedic value)
Scroll to the very bottom to see a video of it in action.
Goal
This model is intended to be trained with all active members of parliament so it can be applied to parliamentlive.tv to monitor all the activity in the House of Commons and House of Lords.
We don't think our tax dollars should be used to pay someone to sleep, do you?
Development Journey
Algorithm Flowchart
Face Detecting with dlib
Relying on dlib Convolutional Neural Network's deep metric learning we are able to identify faces in the frame using a Histogram of Oriented Gradients. In this method, the transitions of darkness in the color of adjacent pixels are mapped as gradients, simplifying the image making it easier to isolate the faces from the frame.
Face Encoding with dlib
We encode photographs of Politician's faces that we want the model to recognize into an output vector with 128 values for each face. The faces are then saved to an array which later referenced when identifying
Drowsiness Detection with OpenCV's Haar cascades
Closed eyes are detected leveraging the facial landmark data and extracting the eye perimeter made up of six points. The Eye Aspect Ratio formula is used to determine the relationship between width and height of the eyes, zero being when the eyes are closed. This is an example of classification using Support Vector Machine, which is a much faster approach than image processing the whites of the eyes. A threshold of time was set to distinguish sleeping from blinking.
Twitter Demo
For demonstration purposes during the presentation we set up the webcam to recognize, detect, and tweet. Here is a screen grab of an executed example on our twitter.
Testing Twitter API and webcam
Here is a recording of the webcam detecting, recognizing, and tweeting.
Final Video Presentation