Project Blog

CS298

Week 12: May 4 - May 11

Minutes

• Submitted the final version of CS298 Project Report
• Finalized the CS298 defense slides
• Demonstrated end to end working of application

To Do

• CS 298 Defense

Week 12: Apr 27 - May 4

Minutes

• Successfully implemented Information Fusion Algorithm in application
• Evaluated initial draft of CS298 Project report
• Discussed about CS298 Defense Slides

To Do

• Finalize the CS298 Project report
• CS298 Defense Slides

Week 11: Apr 20 - Apr 27

Minutes

• Finalized Information Fusion technique (Fusion Algorithm) to calculate final emotion
• Discussed about CS298 final project report

To Do

• Implement information fusion algorithm in the application
• Start writing the initial draft of the report

Week 10: Apr 13 - Apr 20

Minutes

• Evaluated Information Fusion technique (Fusion Algorithm) based on multiple test cases
• Successfully calculated MFCCs using JLibrosa library
• Successfully deployed Speech Emotion recognition model on Android application

To Do

• Continue to research further about Information Fusion
• Test the application to predict facial and speech emotion

Week 9: Apr 6 - Apr 13

Minutes

• Discussed about Information Fusion technique (Fusion Algorithm) to calculate final emotion of the overall conversation
• Successfully installed JLibrosa library in the project

To Do

• Continue to research further about Information Fusion
• Calculate MFCCs using JLibrosa library

Week 8: Mar 30 - Apr 6

Minutes

• Discussed about deploying Speech Emotion Recognition tflite model on application
• Evaluated performance of the application
• Discussed about Android JLibrosa library to calculate MFCC

To Do

• Continue working on Android application
• Research about how to predict final emotion based on speech and face
• Install JLibrosa library in the project

Week 7: Mar 23 - Mar 30

Minutes

• Delivered the improved Speech Emotion Recognition Model
• Finalized which speech feature (MFCC) to use for Speech Emotion Recognition

To Do

• Continue working on Android application
• Convert the Keras Model to tflite model

Week 6: Mar 16 - Mar 23

Minutes

• Delivered initial Speech Emotion Recognition Model
• Discussed about the dependencies required to provide voice record feature in the application
• Reviewed the model architecture and discussed about improving the accuracy

To Do

• Continue working on Android application to provide speech support
• Continue to work on Speech Emotion Recognition Model to improve accuracy

Week 5: Mar 9 - Mar 16

Minutes

• Update regarding the Speech Emotion Recognition model
• Discussed about how to add a voice record feature in the application
• Discussed about converting the Keras model to tflite model

To Do

• Continue working on Android application to provide speech support
• Continue to develop the initial model to detect emotion based on speech

Week 4: Mar 2 - Mar 9

Minutes

• Explained the significance of MFCC, Chroma and Mel Spectrogram
• Demo of Android application with the functionality of recognizing emotion from both cameras
• Finalized the RAVDESS dataset for training and test the Speech Emotion Recognition Model

To Do

• Continue working on Android application to provide speech support
• Develop the initial model to detect emotion based on speech

Week 3: Feb 23 - Mar 2

Minutes

• Discussed about Speech Features (MFCC, Chroma, Mel Spectrogram)
• Android application update regarding frame captures
• Discussed about the dataset to use for Speech Emotion Recognition

To Do

• Continue working on Android application
• Understand the significance and weightage of each feature discussed in the meeting

Week 2: Feb 16 - Feb 23

Minutes

• Discussed about the speech recognition model and features of data
• Discussed about hybrid approach and decided the features to use for emotion recognition

To Do

• Continue working on Android application to capture frames
• Research about papers explaining the recognition of emotion using speech

Week 1: Feb 9 - Feb 16

Minutes

• Kick off meeting and further discussion of idea
• CS298 Proposal discussion and approval

To Do

• Continue working on Android application to capture frames

CS297

Week 11 : Dec 2- Dec 8

Minutes

• Final Report Submission and CS297 guidelines
• Possible Improvements in the CS297 Report
• CS298 initial plan

To Do

• Validate website with required guidelines

Week 10 : Nov 24- Dec 2

Minutes

• Deployed tflite model on android application for initial testing
• Evaluated CS297 final report and discussed about possible improvements

To Do

• CS297 Report final draft

Week 9 : Nov 11- Nov 24

Minutes

• Delivered Android Application that can access both cameras simultaneously (Deliverable 3)
• Initial Discussion about Deliverable 4
• Discussed about converting Keras model to tfite model

To Do

• Deliverable 4
• Converted trite model to deploy on android application

Week 8 : Nov 11- Nov 18

Minutes

• Delivered Initial Android Application
• Discussed about further scope of Deliverable 3

To Do

• Deliverable 3 Due

Week 7 : Nov 4- Nov 11

Minutes

• Submitted Deliverable 2 with 51 Videos Dataset
• Improvements in Unity Video Using Camera Move
• Discussed about Deliverable 3

To Do

• Deliverable 2
• Initial phase of Deliverable 3

Week 6 : Oct 27- Nov 4

Minutes

• Delivered Initial Video of Facial Expression Generated by Unity
• Discussed Changes to make it better

To Do

• Deliverable 2
• Improvement in Video

Week 5 : Oct 20- Oct 27

Minutes

• Delivered a basic unity facial expression model
• Discussed Unity to generate dataset to change expressions

To Do

• Deliverable 2
• Presentation on Unity Basics

Week 4 : Oct 13- Oct 20

Minutes

• Delivered depth wise sequential model
• Delivered first deliverable
• Discussed Unity to generate dataset

To Do

• Do a Tutorial on Unity
• Initial Unity Model to show one expression

Week 3 : Oct 6- Oct 13

Minutes

• First Paper Presentation. Paper Name : "Research of Facial Expression Recognition Based on Deep Learning"
• Finalized the Sequential Model
• Discussed about Depth wise sequential model described in the paper and its advantages

To Do

• Design of Depth wise sequential model for my application

Week 2 : Sept 29- Oct 6

Minutes

• Discussed about the FerNET2013 Dataset. Categorized the Expressions in 7 classes
• Discussed the accuracy of initial sequential Model

To Do

• First Paper Presentation
• Parameter Tuning in Sequential Model

Week 1 : Sept 23- Sept 29

Minutes

• Kickoff meeting and Idea Discussion
• Finalized the Proposal

To Do

• Finalizing the Sample Dataset
• Image Preprocessing
• Design initial Sequential Model