Chris Pollett >
Students > [Bio] [Blog] [Depth Wise Convolutional Model-PDF] |
CS298 ProposalSign Language Assistant.Akshay Kajale (akshay.kajale@sjsu.edu) Advisor: Dr. Chris Pollett Committee Members: Dr. Robert Chun, Mr. Kiran Salte Abstract: Emotion Recognition is one of the most researched topics in modern day machine learning arena. There are various ways in which emotion can be recognized. For example, facial expressions, body postures, speech tone etc. The focus of this research project is to develop a prototype of emotion recognition system by making a hybrid model using computer vision and natural language processing techniques. Our goal hybrid system would use video feeds of different facial expressions and speech features to recognize emotions. Finally, the machine learning model will be deployed on Android application to predict the emotion. The application can access both front and back camera simultaneously and can capture the audio features of the person talking in front of cameras. The hybrid model is implemented with the help of neural network. Our prototype will operate on videos created in Unity 3D humanoid model performing different facial expressions. CS297 Results:
Proposed Schedule:
Key Deliverables:
Innovations and Challenges:
References [1]L. Zhang, Y. Yang, W. Li, S. Dang and M. Zhu, "Research of Facial Expression Recognition Based on Deep Learning," 2018 IEEE 9th International Conference on Software Engineering and Service Science (ICSESS), Beijing, China, 2018, pp. 1-4, doi: 10.1109/ICSESS.2018.8663777. [2]Mao Xu, Wei Cheng, Qian Zhao, Li Ma and Fang Xu, "Facial expression recognition based on transfer learning from deep convolutional networks," 2015 11th International Conference on Natural Computation (ICNC), Zhangjiajie, 2015, pp. 702-708, doi: 10.1109/ICNC.2015.7378076. [3]A. Fathallah, L. Abdi and A. Douik, "Facial Expression Recognition via Deep Learning," 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA), Hammamet, 2017, pp. 745-750, doi: 10.1109/AICCSA.2017.124. [4]D. Kalita, "Designing of Facial Emotion Recognition System Based on Machine Learning," 2020 8th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), Noida, India, 2020, pp. 969-972, doi: 10.1109/ICRITO48877.2020.9197771. [5] L. B. Letaifa, M. I. Torres and R. Justo, "Adding dimensional features for emotion recognition on speech," 2020 5th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), Sousse, Tunisia, 2020, pp. 1-6, doi: 10.1109/ATSIP49331.2020.9231766. [6] Ekman P. Darwin's contributions to our understanding of emotional expressions. Philos Trans R Soc Lond B Biol Sci. 2009;364(1535):3449-3451. doi:10.1098/rstb.2009.0189 |