Human emotions have a long evolutionary purpose for our survival as a species. They are either a reaction to an external stimulus, or a spontaneous expression of an internal thought process. Without emotions we would not have survived as a species , substantiated by the fact that our intelligence has improved as a result of our emotions. We have now come to a point where we cannot detach our emotions from our decisions.
Considered three major signals to identify the emotional state of the user:
- Facial Expression : Using deep learning (CNN) to match the users expressions to the various moods encountered by a human being.
- Tonality of Speech :Measured pitch, frequency of the user’s voice to gauge his emotions with the help of RNN.
- Contextual Information: The context provided by the user about his life ( daily activities ) is also crucial to recognize the underlying causes of his emotional state. So the chatbot will create personalized conversations to explore the user’s daily activities.
These factors helped us to narrow down on the emotional state of the user (like joy, sadness, surprise, disappointment, fear, and anger). The bot would then engage in conversation to help the user eradicate negative feelings and improve his/her emotional state which would have otherwise clouded his/her judgement.