Automatically computing emotions aroused from images through shape modeling
Shape features in natural images influence emotions aroused in human beings. An in-depth statistical analysis helps to understand the relationship between shapes and emotions. Through experimental results on the International Affective Picture System (IAPS) dataset, evidence is presented as to the significance of roundness-angularity and simplicity-complexity on predicting emotional content in images. Shape features are combined with other state-of-the-art features to show a gain in prediction and classification accuracy. Emotions are modeled from a dimensional perspective in order to predict valence and arousal ratings, which have advantages over modeling the traditional discrete emotional categories. Images are distinguished vis-a-vis strong emotional content from emotionally neutral images with high accuracy. All of the methods and steps disclosed herein are implemented on a programmed digital computer, which may be a stand-alone machine or integrated into another piece of equipment such as a digital still or video camera including, in all embodiments, portable devices such as smart phones.
1. A system for automatically predicting emotional content of a digital image, comprising:
a programmed computer having memory and an input for receiving the digital image from the memory of a device used to capture or store the image, and wherein the computer is programmed to perform the following functions automatically:
(a) extract visual information from the image;
(b) model shape features in the image including roundness and angularity based upon the extracted visual information; and
(c) compute information relating to the emotional content of the image based upon the extracted visual information and modeled shape features; and
an output device in communication with the programmed computer enabling a user to receive the information relating to the computed emotional content of the image, the information including a classification of the image into one of a plurality of categories at least including anger, disgust, fear, sadness, amusement, awe, contentment, and excitement,
wherein the roundness in the image is automatically computed by fitting the shape features extracted from the image to a circle, and
wherein the angularity in the image is automatically computed by fitting the shape features extracted from the image to ellipses and measuring the elongatedness of the fitted ellipses.
2. The system of claim 1 , wherein:
the programmed computer also receives non-visual information about the image; and
the computer is further operative to automatically output information regarding the emotional content of the image based upon the modeled shape features and one or both of the extracted visual features and non-visual information.
3. The system of claim 2 , wherein:
the programmed computer forms part of a camera; and
the non-visual information includes camera setting parameters.
4. The method of claim 2 , further including:
an input for receiving textual information about the image; and
wherein the non-visual information includes the textual information about the image.
5. The system of claim 1 , wherein the image received by the input is one of a still digital image or a plurality of sequential digital images representing frames in a video or a movie.
6. The system of claim 1 , wherein the computer is further programmed to automatically perform the following functions:
use the modeled shape features to determine valence or arousal coordinates based upon a dimensional space model of emotions with valence and arousal coordinates; and
compute the emotional content of the image based upon the valence or arousal coordinates of the image in the space model.
7. The system of claim 1 , wherein the computer is further programmed to automatically use the shape features to distinguished images with strong emotional content from emotionally neutral images.
8. The system of claim 1 , wherein the computer is further programmed to automatically model simplicity based upon on one or more shapes extracted from the image; and
compute the emotional content based upon the roundness, angularity or simplicity.
9. The system of claim 1 , wherein the computer is further programmed to automatically perform the following functions:
identify contours in the image;
represent the contours as lines and curves;
perform statistical analyses on the lines and curves to model the visual properties of roundness, angularity, and simplicity; and
compute the emotional content based upon the roundness, angularity and simplicity.
10. The system of claim 9 , wherein the computer is programmed to determine if the contours are continuous.