Facial Recognition: Tic-tac-toe

Per the rules of the University of California, Santa Cruz, the source code for this project can not be shared publicy. To view the source code please send me an email, which can be found in the contact section of the website, and state a reason for requesting source code.

Introduction

In this project, I explored the innovative application of deep learning to create a novel game controller by recognizing facial expressions and using them as inputs for gameplay. My collaborator, David Amaya, and I completed this project in 2 weeks for a Game AI course at UC, Santa Cruz.

Analysis

Initial Model Development

We utilized Keras and TensorFlow to build a classifier capable of distinguishing between neutral, happy, and surprised facial expressions. The initial model, trained with a dataset provided by Kaggle, incorporated convolutional and max pooling layers, fully connected layers, and a softmax activation function.

Model Training and Optimization

Throughout the training process, we monitored accuracy and loss metrics to refine the model and prevent overfitting. The successfully optimized model demonstrated by our enhanced version achieved improved validation performance.

Gameplay Integration

Using the trained model, we developed a gameplay interface that leveraged facial expressions to dictate moves in a tic-tac-toe game. Despite challenges with real-world accuracy, the project provided valuable insights into the model's responsiveness and potential improvements.

Transfer Learning

We explored transfer learning by adapting the facial recognition model to classify between two completely different categories, demonstrating the model's adaptability and broad applicability.

Conclusion

This project highlights the practical use of deep learning in creating interactive game interfaces and sets the stage for further exploration into innovative user-interface technologies. Through ongoing refinement, we enhanced the accuracy and effectiveness of our models, further exploring the practical applications and interconnections of deep learning and gaming.

Sources

Cover Image: https://www.g2.com/articles/facial-recognition