Image-capturing apparatus and method, facial expression evaluation apparatus, and program
View Patent ↗An image-capturing apparatus for capturing an image by using a solid-state image-capturing device may include a face detector configured to detect a face of a human being on the basis of an image signal in a period until an image signal obtained by image capturing is recorded on a recording medium; an expression evaluation section configured to evaluate the expression of the detected face and to compute an expression evaluation value indicating the degree to which the detected face is close to a specific expression in relation to expressions other than the specific expression; and a notification section configured to notify notification information corresponding to the computed expression evaluation value to an image-captured person.
1. An image-capturing apparatus for capturing an image using a solid-state image-capturing device, the image-capturing apparatus comprising:
a face detector configured to detect a face of a human being based on an image signal in a period until an image signal obtained by image capturing is recorded on a recording medium;
an expression evaluation section configured to evaluate the expression of the detected face and to compute an expression evaluation value indicating the degree to which the detected face is closer to a specific expression than to expressions other than the specific expression; and
a notification section configured to output notification information corresponding to the computed expression evaluation value,
wherein the expression evaluation section computes the expression evaluation value indicating the degree to which the detected face is closer to a first face group based on a plurality of pieces of face data associated with the specific expression than to a second face group based on a plurality of pieces of face data associated with the expressions other than the specific expression, and
the expression evaluation section has pre-stored therein information on a determination axis obtained by performing linear discriminant analysis based on a first group of sample images of a plurality of faces in which each of the sample images has the specific expression and a second group of sample images of the plurality of faces in which each of the sample images has the expressions other than the specific expression, and computes the expression evaluation value based on the magnitude of projection components of a vector based on an image signal of the detected face with respect to a vector on the determination axis.
2. The image-capturing apparatus according to claim 1 , wherein the notification section is configured to output notification information corresponding to the computed expression evaluation value to an image-captured person.
3. The image-capturing apparatus according to claim 1 , wherein the determination axis is computed based on data such that data of a vector based on the sample images is dimension-compressed by principal component analysis.
4. The image-capturing apparatus according to claim 3 , wherein the expression evaluation section has pre-stored therein data of a vector on the determination axis as data of a vector having a dimension before being dimension-compressed, and computes the expression evaluation value by computing an inner product of the vector having a dimension before being dimension-compressed and the vector based on the image signal of the detected face.
5. The image-capturing apparatus according to claim 1 , wherein the expression evaluation section normalizes the image signal of the detected face to an image size of the sample images used when the determination axis is determined, and computes the expression evaluation value using the normalized image signal.
6. The image-capturing apparatus according to claim 5 , wherein the expression evaluation section performs, on the normalized image signal, a masking process for masking a signal in an image region that does not substantially influence the expression determination, and computes the expression evaluation value using the masked image signal.
7. An image-capturing method of capturing an image using a solid-state image-capturing device, the method comprising:
detecting a face of a human being based on an image signal in a period until an image signal obtained by image capturing is recorded on a recording medium;
evaluating the expression of the detected face and to compute an expression evaluation value indicating the degree to which the detected face is closer to a specific expression than to expressions other than the specific expression; and
outputting notification information corresponding to the computed expression evaluation value,
wherein the expression evaluating step includes computing the expression evaluation value indicating the degree to which the detected face is closer to a first face group based on a plurality of pieces of face data associated with the specific expression than to a second face group based on a plurality of pieces of face data associated with the expressions other than the specific expression, and
the expression evaluating step includes computing the expression evaluation value based on the magnitude of projection components of a vector based on an image signal of the detected face with respect to a vector on a determination axis using pre-stored information on the determination axis obtained by performing linear discriminant analysis based on a first group of sample images of a plurality of faces in which each of the sample images has the specific expression and a second group of sample images of the plurality of faces in which each of the sample images has the expressions other than the specific expression, and.
8. The image-capturing method according to claim 7 , wherein the outputting notification information step outputs notification information corresponding to the computed expression evaluation value to an image-captured person.
9. The image-capturing method according to claim 7 , wherein the determination axis is computed based on data such that data of a vector based on the sample images is dimension-compressed by principal component analysis.
10. The image-capturing method according to claim 9 , further comprising pre-storing data of a vector on the determination axis as data of a vector having a dimension before being dimension-compressed, and the expression evaluating step includes computing the expression evaluation value by computing an inner product of the vector having a dimension before being dimension-compressed and the vector based on the image signal of the detected face.
11. The image-capturing method according to claim 7 , wherein the expression evaluating step includes normalizing the image signal of the detected face to an image size of the sample images used when the determination axis is determined, and computing the expression evaluation value using the normalized image signal.
12. The image-capturing method according to claim 11 , wherein the expression evaluating step includes performing, on the normalized image signal, a masking process for masking a signal in an image region that does not substantially influence the expression determination, and computing the expression evaluation value using the masked image signal.