Gesture recognition method and apparatus, and storage medium
View Patent ↗Provided are a gesture recognition method and apparatus, and a storage medium. The method includes: obtaining a first feature map corresponding to a target region in a depth image, and determining a target key point from the first feature map; establishing a three-dimensional coordinate system in the first feature map by using the target key point, and predicting other three-dimensional coordinates of other key points in the three-dimensional coordinate system, the other key points being key points other than the target key point in the first feature map; and determining, based on the target key point and the other three-dimensional coordinates, a plurality of absolute coordinates of a plurality of key points, to determine a gesture of a target recognition object in the target region based on the plurality of absolute coordinates, the plurality of key points including the target key point and the other key points.
1. A gesture recognition method, comprising:
obtaining a first feature map that is a feature map corresponding to a target region in a depth image, and determining a target key point from the first feature map, the target recognition object being a hand, and the target key point being a key point of a thumb root;
establishing a three-dimensional coordinate system in the first feature map by taking the target key point as a coordinate origin, and predicting other three-dimensional coordinates of other key points in the three-dimensional coordinate system, the other key points being key points other than the target key point in the first feature map; and
determining, based on the target key point and the other three-dimensional coordinates, a plurality of absolute coordinates of a plurality of key points, to determine a gesture of a target recognition object in the target region based on the plurality of absolute coordinates, the plurality of key points comprising the target key point and the other key points.
2. The method according to claim 1 , wherein said predicting the other three-dimensional coordinate of the other key points in the three-dimensional coordinate system comprises:
inputting the first feature map into an XYZ encoder and a fully connected layer sequentially, to obtain the other three-dimensional coordinates corresponding to the other key points based on the three-dimensional coordinate system.
3. The method according to claim 1 , wherein said determining, based on the target key point and the other three-dimensional coordinates, the plurality of absolute coordinates of the plurality of key points comprises:
determining target absolute coordinates of the target key point; and
determining, based on the target absolute coordinates and the other three-dimensional coordinates, the plurality of absolute coordinates.
4. The method according to claim 3 , wherein said determining the target absolute coordinates of the target key point comprises:
obtaining a two-dimensional position of the target key point;
determining, based on the two-dimensional position, a target absolute depth of the target key point; and
determining, based on the target absolute depth of the target key point, the target absolute coordinates of the target key point.
5. The method according to claim 4 , wherein said obtaining the two-dimensional position of the target key point comprises:
inputting the first feature map into a UV encoder to obtain target UV coordinates of the target key point; and
determining the target UV coordinates as the two-dimensional position.
6. The method according to claim 4 , wherein said determining, based on the two- dimensional position, the target absolute depth of the target key point comprises:
in response to determining that the two-dimensional position is located in a background of the depth image, determining depth values of pixels within a predetermined range having the two- dimensional position as a center, and determining a closest depth from the depth values of the pixels within the predetermined range; and
determining the target absolute depth by using the closest depth.
7. A non-transitory storage medium, having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method according to claim 1 .
8. A gesture recognition apparatus, comprising:
a processor; and
a memory, wherein the processor, when executing an operating program stored on the memory, implements a gesture recognition method comprising:
obtaining a first feature map that is a feature map corresponding to a target region in a depth image, and determining a target key point from the first feature map, the target recognition object being a hand, and the target key point being a key point of a thumb root;
establishing a three-dimensional coordinate system in the first feature map by taking the target key point as a coordinate origin, and predicting other three-dimensional coordinates of other key points in the three-dimensional coordinate system, the other key points being key points other than the target key point in the first feature map; and
determining, based on the target key point and the other three-dimensional coordinates, a plurality of absolute coordinates of a plurality of key points, to determine a gesture of a target recognition object in the target region based on the plurality of absolute coordinates, the plurality of key points comprising the target key point and the other key points.
9. The apparatus according to claim 8 , wherein the processor, when executing the operating program stored on the memory, implements:
inputting the first feature map into an XYZ encoder and a fully connected layer sequentially, to obtain the other three-dimensional coordinates corresponding to the other key points based on the three-dimensional coordinate system.
10. The apparatus according to claim 8 , wherein the processor, when executing the operating program stored on the memory, implements:
determining target absolute coordinates of the target key point; and
determining, based on the target absolute coordinates and the other three-dimensional coordinates, the plurality of absolute coordinates.
11. The apparatus according to claim 10 , wherein the processor, when executing the operating program stored on the memory, implements:
obtaining a two-dimensional position of the target key point;
determining, based on the two-dimensional position, a target absolute depth of the target key point; and
determining, based on the target absolute depth of the target key point, the target absolute coordinates of the target key point.
12. The apparatus according to claim 11 , wherein the processor, when executing the operating program stored on the memory, implements:
inputting the first feature map into a UV encoder to obtain target UV coordinates of the target key point; and
determining the target UV coordinates as the two-dimensional position.
13. The apparatus according to claim 11 , wherein the processor, when executing the operating program stored on the memory, implements:
in response to determining that the two-dimensional position is located in a background of the depth image, determining depth values of pixels within a predetermined range having the two- dimensional position as a center, and determining a closest depth from the depth values of the pixels within the predetermined range; and
determining the target absolute depth by using the closest depth.