Chris Pollett > Students >

    ( Print View)



    [CS 297 Proposal]

    [Deliverable 1]

    [Depth Wise Convolutional Model-PDF]

    [Deliverable 2]

    [My Experience with Unity-PDF]

    [Deliverable 3]

    [Deliverable 4]

    [CS297 Final Report-PDF]

    [CS 298 Proposal]

    [CS298 Final Report-PDF]

    [CS298 Slides-PDF]

Deliverable 2

Visual and Lingual Emotional Recognition using Deep Learning Techniques

Unity provides lots of open-source assets to try your hands on. I used an open-source Anime Character provided by unity to create my synthetic dataset that will be used for initial testing of my model. We created the animations of the seven emotions that Anime character could express. In the deliverable, the scope of the dataset is limited to 51 videos, each of 20 seconds. The scene created in Unity is captured from different angle with character expressing random expressions to improve the variety in the dataset. In the scene, the Humanoid will express different expressions randomly and the camera angle will be changed after every 20 seconds. This dataset will be used for the initial training and testing of the model in my project. Model will capture the distinct facial features to predict the emotion.

Introduction to Unity

  • Cross platform game engine developed by Unity.
  • Create immersive 3D experiences for real world applications at scale.
  • Majorly used in game development and 3D animations.
  • Some Popular games made in unity: Temple run, The long dark, Kerbal space program.
  • Popular language used in Unity for Scripting: C#

My Experience with Unity

  • I created a 3D character in Unity which changes its facial expressions randomly.
  • Total 7 Expressions are covered: Angry, Disgust, Fear, Happy, Sad, Surprised and Neutral
  • Used C# for Scripting for randomly selecting expressions to display
  • I also created a 3D game in which you can throw a cube in 3D and set it up back to their original position based on user's command.

Unity Character

  • Error loading image

Facial Expressions of Model

  • Neutral

  • Error loading image
  • Happy

  • Error loading image
  • Sad

  • Error loading image
  • Error loading image
  • Smile

  • Error loading image
  • Fear

  • Error loading image
  • Angry

  • Error loading image

Animator Controller

This animator controller is used to design the flow of the Expressions. This controlled is linked with C# Script to select random expressions

  • Error loading image


This Dataset contains total 51 Videos. Each video is of 20 Sec and its taken from different camera angles to cover maximum scope of Expressions.

Download Dataset

Source Code