Facial Recognition Wearable
For my senior year capstone project for the Intelligent Systems Practicum, my team built and tested a wearable device that could aid the visually impaired with recognizing faces in social contexts.
The device is capable of identifying and storing new faces based on images taken, as well as recognizing a previously encountered face and supplying social information including name, date last encountered, and other relevant information.
Momento
Helping you recognize and remember faces you've encountered before
That's me!
Team Momento:
Diana, Birthe, Jenny, Qianhui, Bryan
Training the Algorithm
I led the facial recognition algorithm training and development. We used OpenCV to help the model with image processing and imported modules from the framework. Each incoming image had to be applied OpenCV’s face detection, which notes the presence of a face in the image, then extracts 128-d embeddings to help quantify each face. By doing so, I made sure that we have a quantifiable way of identifying, sorting, and categorizing an abstract image of a face. I also coded in a k-fold cross-validation algorithm to evaluate results, using 5 images of the same face and one different face to cross-validate.
A Raspberry Pi Arducam 5MP Camera was used
to capture the images in real-time
Building the Feature Vectors
To help with the facial identification and categorization process, I built a “Face” object that hosted attributes such as “height” “width” “color” etc. These dimensions were then converted into numerical representations of the face by having each element in the training set mapped onto a feature vector comprising of integer or string values representing the facial features. To automate this process, I developed an algorithm that derived the feature vectors in real-time as it detects faces and receive input.
The Raspberry Pi is very portable and easily
attachable to a range of clothings
With the upgraded WiFi and Bluetooth capabilities, we were able to migrate the database to an offline server and reduce overheating
Matching Faces
Working with another teammate of mine, we also created a matching algorithm to determine the similarity between the feature vectors I built out. The facial analysis used the library model VGG-Face and it imported pre-trained weights and deep learning networks. Distance values were assigned to indicate the numerical difference between two faces. Once all the training data is prepared (using imagery data collected from our consented classmates), I helped program the Rapsberry Pi to interact with the facial recognition API. This process included the process of assigning unique face_tokens and user_ids for each image received.
Takeaways
This project brought me an abundance of takeaways. It not only offered a practical application to my technical skills in algorithm design and development (which was definitely a great challenge for me at the time, but deeply appreciated). It also developed further my team collaboration and communication abilities, as well as task delegation skills as the team leader. I learned to develop a full machine learning product, from ideation, design, implementation, to a testable wearable (which garnered even the attention of the Mayor of Chicago herself when she visited our Demo Day). I am deeply proud of and thankful for Team Momento and our EECS 338 Practicum professor, Kris J Hammond, as well as our many teaching assistants who offered us valuable insights on the way.