Expression estimation device, control method, control program, and recording medium
An expression estimation device configured to estimate an expression of a person from an image has an expression determination unit that analyzes the image of the person to generate first expression information indicating an expression of the person, a motion determination unit that analyzes the image of the person to determine a type of motion of the person, and an expression estimation unit that generates second expression information indicating the expression of the person according to the first expression information generated by the expression determination unit and the type of motion determined by the motion determination unit.
1. An expression estimation device configured to estimate an expression of a person from an image, the expression estimation device comprising:
an expression determination unit that analyzes the image of the person to generate first expression information indicating an expression of the person;
a motion determination unit that analyzes the image of the person to determine a type of motion of the person; and
an expression estimation unit that generates second expression information indicating the expression of the person according to the first expression information generated by the expression determination unit and the type of motion determined by the motion determination unit,
wherein the first expression information is information in which a fitness for each of the different expressions is indicated in terms of numerical value, and
wherein the expression estimation unit generates the second expression information by correcting the numerical value of the first expression information using predetermined correction information on each type of motion.
2. The expression estimation device according to claim 1 , further comprising:
a motion registration unit that stores the motion assigned in the image while correlating the type of motion and the correction information on the motion with each other.
3. The expression estimation device according to claim 1 , further comprising:
an attribute determination unit that analyzes the image to determine which predetermined attribute corresponds to the person,
wherein the expression estimation unit generates the second expression information by correcting the numerical value of the first expression information using the predetermined correction information on each type of motion, and
wherein the correction information corresponds to the attribute determined by the attribute determination unit.
4. The expression estimation device according to claim 1 , further comprising:
a fitness proximity determination unit that determines whether a difference between the fitness having the largest numerical value and the fitness having the second largest numerical value in the first expression information is greater than or equal to a predetermined upper limit,
wherein the expression estimation unit generates the first expression information as the second expression information without using the correction information when the fitness proximity determination unit determines that the difference is greater than or equal to the upper limit, and
wherein the expression estimation unit generates the second expression information using the correction information when the fitness proximity determination unit determines that the difference is less than the upper limit.
5. The expression estimation device according to claim 1 , further comprising:
a fit expression determination unit that determines whether the expression in which the fitness has the largest numerical value in the first expression information is matched with a predetermined expression, and
wherein the fit expression determination unit decides whether the motion determination unit determines the type of motion according to a result of the determination,
wherein the expression estimation unit generates the second expression information using the correction information when the motion determination unit determines the type of motion, and
wherein the expression estimation unit generates the first expression information as the second expression information without using the correction information when the motion determination unit does not determine the type of motion.
6. The expression estimation device according to claim 2 , further comprising:
an attribute determination unit that analyzes the image to determine which predetermined attribute corresponds to the person,
wherein the expression estimation unit generates the second expression information by correcting the numerical value of the first expression information using the predetermined correction information on each type of motion, and
wherein the correction information corresponds to the attribute determined by the attribute determination unit.
7. The expression estimation device according to claim 2 , further comprising:
a fitness proximity determination unit that determines whether a difference between the fitness having the largest numerical value and the fitness having the second largest numerical value in the first expression information is greater than or equal to a predetermined upper limit,
wherein the expression estimation unit generates the first expression information as the second expression information without using the correction information when the fitness proximity determination unit determines that the difference is greater than or equal to the upper limit, and
wherein the expression estimation unit generates the second expression information using the correction information when the fitness proximity determination unit determines that the difference is less than the upper limit.
8. The expression estimation device according to claim 3 , further comprising:
a fitness proximity determination unit that determines whether a difference between the fitness having the largest numerical value and the fitness having the second largest numerical value in the first expression information is greater than or equal to a predetermined upper limit,
wherein the expression estimation unit generates the first expression information as the second expression information without using the correction information when the fitness proximity determination unit determines that the difference is greater than or equal to the upper limit, and
wherein the expression estimation unit generates the second expression information using the correction information when the fitness proximity determination unit determines that the difference is less than the upper limit.
9. The expression estimation device according to claim 2 , further comprising:
a fit expression determination unit that determines whether the expression in which the fitness has the largest numerical value in the first expression information is matched with a predetermined expression, and
wherein the fit expression determination unit decides whether the motion determination unit determines the type of motion according to a result of the determination,
wherein the expression estimation unit generates the second expression information using the correction information when the motion determination unit determines the type of motion, and
wherein the expression estimation unit generates the first expression information as the second expression information without using the correction information when the motion determination unit does not determine the type of motion.
10. The expression estimation device according to claim 3 , further comprising:
a fit expression determination unit that determines whether the expression in which the fitness has the largest numerical value in the first expression information is matched with a predetermined expression, and
wherein the fit expression determination unit decides whether the motion determination unit determines the type of motion according to a result of the determination,
wherein the expression estimation unit generates the second expression information using the correction information when the motion determination unit determines the type of motion, and
wherein the expression estimation unit generates the first expression information as the second expression information without using the correction information when the motion determination unit does not determine the type of motion.
11. A method for controlling, via computer, an expression estimation device configured to estimate an expression of a person from an image, the control method comprising:
acquiring the image from an image capturing device;
analyzing the image to generate first expression information indicating the expression of the person;
analyzing the image to determine a type of motion of the person; and
generating second expression information indicating the expression of the person according to the first expression information and the type of motion,
wherein the first expression information is information in which a fitness for each of the different expressions is indicated in terms of numerical value, and
wherein the expression estimation unit generates the second expression information by correcting the numerical value of the first expression information using predetermined correction information on each type of motion.
12. A control program stored on a non-transitory computer-readable medium configured to estimate an expression of a person from an image, the control program causing a computer to perform:
analyzing the image to generate first expression information indicating the expression of the person;
analyzing the image to determine a type of motion of the person; and
generating second expression information indicating the expression of the person according to the first expression information and the type of motion,
wherein the first expression information is information in which a fitness for each of the different expressions is indicated in terms of numerical value, and
wherein the expression estimation unit generates the second expression information by correcting the numerical value of the first expression information using predetermined correction information on each type of motion.