Gesture recognition device, operation method for gesture recognition device, and operation program for gesture recognition device

1 . A gesture recognition device that recognizes, based on an image obtained by imaging a person to be imaged with a digital camera having an imaging range changing mechanism for changing an imaging range, a gesture represented by a hand of the person to be imaged, the gesture recognition device comprising:

at least one processor,

wherein the at least one processor is configured to:

set an area of the image which includes a face of the person to be imaged and determine a center position of the area of the image,

generate, in a case where the center position deviates from a center line of a long side of the image, a movement control signal to move the digital camera to align the center line of the image with the center position to capture a proper face position image with the position of the face as the set position,

determine whether the hand detected from the proper face position image is a right hand or a left hand based on a positional relationship between a position of the detected hand and the set position,

determine a hand detection region having a center of the hand from the proper face position image,

cut out a rectangular region surrounding the detected hand from the proper face position image to generate a hand region image by matching the center of the hand with a center of the rectangular region, wherein the hand region image includes the rectangular region which has more margin for the detected hand than the hand detection region,

recognize the gesture based on the hand region image,

determine whether a size of the hand region image is larger and whether the size of the hand region is smaller than a predetermined set size,

perform size change processing of enlarging the size of the hand region image to the predetermined set size in response to the size of the hand region image being smaller than the predetermined size, and

perform size change processing of reducing the size of the hand region image to the predetermined size in response to the size of the hand region image being larger than the predetermined size.

2 . The gesture recognition device according to claim 1 , wherein the at least one processor is configured to:

perform mirror-image inversion processing on the proper face position image in a case where the detected hand is not a predetermined one hand of the right hand or the left hand, and

recognize the gesture based on the proper face position image or the proper face position image subjected to the mirror-image inversion processing.

3 . The gesture recognition device according to claim 1 , wherein the at least one processor is configured to:

determine that the position of the detected hand is the right hand in a case where the position of the detected hand is on a left side of the set position and is the left hand in a case where the position of the detected hand is on a right side of the set position, as viewed from an imaging direction of the digital camera.

4 . The gesture recognition device according to claim 1 , wherein the at least one processor is configured to:

generate a joint image obtained by extracting a joint of the hand from the hand region image, and

recognize the gesture based on the joint image.

5 . The gesture recognition device according to claim 4 ,

wherein the at least one processor is configured to:

use a first machine learning model that uses the hand region image subjected to the size change processing as input data and the joint image as output data.

6 . The gesture recognition device according to claim 5 ,

wherein the at least one processor is configured to:

perform mirror-image inversion processing on the proper face position image in a case where the detected hand is not a predetermined one hand of the right hand or the left hand, and

recognize the gesture based on the proper face position image or the proper face position image subjected to the mirror-image inversion processing,

wherein the first machine learning model is a model in which the hand region image showing the predetermined one hand whose size is the set size is selectively provided for learning as input data for learning.

7 . The gesture recognition device according to claim 5 ,

wherein the at least one processor is configured to:

perform mirror-image inversion processing on the proper face position image in a case where the detected hand is not a predetermined one hand of the right hand or the left hand, and

recognize the gesture based on the proper face position image or the proper face position image subjected to the mirror-image inversion processing,

wherein the first machine learning model is a model in which a plurality of the hand region images showing the predetermined one hand whose size is the set size in different postures are selectively provided for learning as input data for learning.

8 . The gesture recognition device according to claim 4 , wherein the at least one processor is configured to:

use a second machine learning model that uses the joint image as input data and a type of the gesture as output data.

9 . The gesture recognition device according to claim 8 ,

wherein the at least one processor is configured to:

perform mirror-image inversion processing on the proper face position image in a case where the detected hand is not a predetermined one hand of the right hand or the left hand,

recognize the gesture based on the proper face position image or the proper face position image subjected to the mirror-image inversion processing, and

perform size change processing of changing a size of the hand region image to set a size of the hand shown in the hand region image to a predetermined set size,

wherein the second machine learning model is a model in which the joint image generated from the hand region image showing the predetermined one hand whose size is the set size is selectively provided for learning as input data for learning.

10 . The gesture recognition device according to claim 8 ,

wherein the at least one processor is configured to:

perform mirror-image inversion processing on the proper face position image in a case where the detected hand is not a predetermined one hand of the right hand or the left hand,

recognize the gesture based on the proper face position image or the proper face position image subjected to the mirror-image inversion processing, and

perform size change processing of changing a size of the hand region image to set a size of the hand shown in the hand region image to a predetermined set size,

wherein the second machine learning model is a model in which a plurality of the joint images generated from a plurality of hand region images showing the predetermined one hand whose size is the set size in different postures are selectively provided for learning as input data for learning.

11 . An operation method for a gesture recognition device for recognizing, based on an image obtained by imaging a person to be imaged with a digital camera having an imaging range changing mechanism for changing an imaging range, a gesture represented by a hand of the person to be imaged, the operation method comprising:

setting an area of the image which includes a face of the person to be imaged and determine a center position of the area of the image;

generating, in a case where the center position deviates from a center line of a long side of the image, a movement control signal to move the digital camera to align the center line of the image with the center position to capture a proper face position image with the position of the face as the set position;

determining whether the hand detected from the proper face position image is a right hand or a left hand based on a positional relationship between a position of the detected hand and the set position;

determining a hand detection region having a center of the hand from the proper face position image;

cutting out a rectangular region surrounding the detected hand from the proper face position image to generate a hand region image by matching the center of the hand with a center of the rectangular region, wherein the hand region image includes the rectangular region which has more margin for the detected hand than the hand detection region;

recognizing the gesture based on the hand region image;

determining whether a size of the hand region image is larger and whether the size of the hand region is smaller than a predetermined set size;

performing size change processing of enlarging the size of the hand region image to the predetermined set size in response to the size of the hand region image being smaller than the predetermined size; and

performing size change processing of reducing the size of the hand region image to the predetermined size in response to the size of the hand region image being larger than the predetermined size.

12 . A non-transitory computer-readable storage medium storing an operation program for a gesture recognition device for recognizing, based on an image obtained by imaging a person to be imaged with a digital camera having an imaging range changing mechanism for changing an imaging range, a gesture represented by a hand of the person to be imaged, the operation program causing a computer to:

set an area of the image which includes a face of the person to be imaged and determine a center position of the area of the image;

generate, in a case where the center position deviates from a center line of a long side of the image, a movement control signal to move the digital camera to align the center line of the image with the center position to capture a proper face position image with the position of the face as the set position; determine whether the hand detected from the proper face position image is a right hand or a left hand based on a positional relationship between a position of the detected hand and the set position;

determine a hand detection region having a center of the hand from the proper face position image,

cut out a rectangular region surrounding the detected hand from the proper face position image to generate a hand region image by matching the center of the hand with a center of the rectangular region, wherein hand region image includes the rectangular region which has more margin for the detected hand than the hand detection region,

recognize the gesture based on the hand region image,

determine whether a size of the hand region image is larger and whether the size of the hand region is smaller than a predetermined set size,

perform size change processing of enlarging the size of the hand region image to the predetermined set size in response to the size of the hand region image being smaller than the predetermined size, and

perform size change processing of reducing the size of the hand region image to the predetermined size in response to the size of the hand region image being larger than the predetermined size.