Action learning method, medium, and electronic device

1. An action learning method, comprising:

acquiring human body moving image data;

determining three-dimensional human body pose action data corresponding to the human body moving image data, wherein the three-dimensional human body pose action data comprising a plurality of three-dimensional human body poses arranged in a chronological order of actions;

matching the three-dimensional human body pose action data with atomic actions in a robot atomic action library to determine robot action sequence data corresponding to the human body moving image data, wherein the robot action sequence data comprises a plurality of robot sub-actions comprising the atomic actions and/or mapped actions obtained by mapping the three-dimensional human body pose action data,

performing action continuity stitching on all robot sub-actions in the robot action sequence data; and

determining a continuous action learned by a robot from the robot action sequence data subjected to the action continuity stitching;

wherein performing action continuity stitching on all robot sub-actions in the robot action sequence data comprises:

performing avoidance processing on self-collision anomaly occurring in the robot action sequence data obtained by the sequential stitching of the robot sub-actions.

2. The method according to claim 1 , wherein determining three-dimensional human body pose action data corresponding to the human body moving image data comprises:

determining two-dimensional human body motion keypoints corresponding to all images in the human body moving image data respectively; and

determining the three-dimensional human body pose action data from two-dimensional keypoint sequence data comprising the two-dimensional human body motion keypoints corresponding to the images respectively.

3. The method according to claim 1 , wherein matching the three-dimensional human body pose action data with atomic actions in a robot atomic action library to determine robot action sequence data corresponding to the human body moving image data comprises:

matching a plurality of human body sub-actions comprised in the three-dimensional human body pose action data in the chronological order of actions sequentially, and determining robot sub-actions corresponding to the human body sub-actions based on the similarity of all atomic actions in the robot atomic action library to the human body sub-actions, wherein the human body sub-action comprises one or more of the three-dimensional human body poses; and

determining the robot action sequence data comprising the robot sub-actions in the chronological order of actions.

4. The method according to claim 3 , wherein determining robot sub-actions corresponding to the human body sub-actions based on the similarity of all atomic actions in the robot atomic action library to the human body sub-actions comprises:

taking the atomic actions with the similarity to the human body sub-action being higher than a similarity threshold as candidate atomic actions under the condition that the human body sub-action is not the first human body sub-action comprised in the three-dimensional human body pose action data and two or more atomic actions with the similarity to the human body sub-action being higher than the similarity threshold exist;

calculating a continuity matching degree between the candidate atomic actions and a robot sub-action corresponding to the previous human body sub-action sequentially; and

determining an atomic action matching the human body sub-action among the candidate atomic actions as the robot sub-action corresponding to the human body sub-action based on the similarity and the continuity matching degree.

5. The method according to claim 4 , wherein determining robot sub-actions corresponding to the human body sub-actions based on the similarity of all atomic actions in the robot atomic action library to the human body sub-actions further comprises:

mapping the human body sub-action to obtain a robot sub-action under the condition that no atomic action with the similarity to the human body sub-action being higher than the similarity threshold exists in the robot atomic action library.

6. The method according to claim 1 , wherein performing action continuity stitching on all robot sub-actions in the robot action sequence data comprises:

performing smooth optimization on robot pose positions and robot motion velocities at joints between adjacent robot sub-actions.

7. The method according to claim 1 , further comprising:

executing the robot action sequence data subjected to the action continuity stitching in a digital twin model of the robot and optimizing the robot action sequence data according to simulation data of the digital twin model;

determining a continuous action learned by a robot from the robot action sequence data subjected to the action continuity stitching comprises:

determining robot action sequence data optimized according to the simulation data of the digital twin model as a continuous action learned by the robot.

8. A non-transitory computer-readable storage medium storing a computer program which, when executed by a processor:

acquires human body moving image data;

determines three-dimensional human body pose action data corresponding to the human body moving image data, wherein the three-dimensional human body pose action data comprising a plurality of three-dimensional human body poses arranged in a chronological order of actions;

matches the three-dimensional human body pose action data with atomic actions in a robot atomic action library to determine robot action sequence data corresponding to the human body moving image data, wherein the robot action sequence data comprises a plurality of robot sub-actions comprising the atomic actions and/or mapped actions obtained by mapping the three-dimensional human body pose action data;

performs action continuity stitching on all robot sub-actions in the robot action sequence data; and

determines a continuous action learned by a robot from the robot action sequence data subjected to the action continuity stitching;

wherein performing action continuity stitching on all robot sub-actions in the robot action sequence data comprises:

performing avoidance processing on self-collision anomaly occurring in the robot action sequence data obtained by the sequential stitching of the robot sub-actions.

9. An electronic device, comprising:

a memory storing a computer program; and a processor, configured to execute the computer program in the memory to: acquire human body moving image data;

determine three-dimensional human body pose action data corresponding to the human body moving image data, wherein the three-dimensional human body pose action data comprising a plurality of three-dimensional human body poses arranged in a chronological order of actions;

match the three-dimensional human body pose action data with atomic actions in a robot atomic action library to determine robot action sequence data corresponding to the human body moving image data, wherein the robot action sequence data comprises a plurality of robot sub-actions comprising the atomic actions and/or mapped actions obtained by mapping the three-dimensional human body pose action data;

perform action continuity stitching on all robot sub-actions in the robot action sequence data; and determine a continuous action learned by a robot from the robot action sequence data subjected to the action continuity stitching;

wherein the processor is further configured to:

perform avoidance processing on self-collision anomaly occurring in the robot action sequence data obtained by the sequential stitching of the robot sub-actions.

10. The electronic device according to claim 9 , the processor is further configured to:

determine two-dimensional human body motion keypoints corresponding to all images in the human body moving image data respectively; and

determine the three-dimensional human body pose action data from two-dimensional keypoint sequence data comprising the two-dimensional human body motion keypoints corresponding to the images respectively.

11. The electronic device according to claim 9 , the processor is further configured to:

match a plurality of human body sub-actions comprised in the three-dimensional human body pose action data in the chronological order of actions sequentially, and determine robot sub-actions corresponding to the human body sub-actions based on the similarity of all atomic actions in the robot atomic action library to the human body sub-actions, wherein the human body sub-action comprises one or more of the three-dimensional human body poses; and

determine the robot action sequence data comprising the robot sub-actions in the chronological order of actions.

12. The electronic device according to claim 11 , the processor is further configured to:

take the atomic actions with the similarity to the human body sub-action being higher than a similarity threshold as candidate atomic actions under the condition that the human body sub-action is not the first human body sub-action comprised in the three-dimensional human body pose action data and two or more atomic actions with the similarity to the human body sub-action being higher than the similarity threshold exist;

calculate a continuity matching degree between the candidate atomic actions and a robot sub-action corresponding to the previous human body sub-action sequentially; and

determine an atomic action matching the human body sub-action among the candidate atomic actions as the robot sub-action corresponding to the human body sub-action based on the similarity and the continuity matching degree.

13. The electronic device according to claim 12 , the processor is further configured to:

map the human body sub-action to obtain a robot sub-action under the condition that no atomic action with the similarity to the human body sub-action being higher than the similarity threshold exists in the robot atomic action library.

14. The electronic device according to claim 9 , the processor is further configured to:

perform smooth optimization on robot pose positions and robot motion velocities at joints between adjacent robot sub-actions.

15. The electronic device according to claim 9 , the processor is further configured to:

execute the robot action sequence data subjected to the action continuity stitching in a digital twin model of the robot and optimizing the robot action sequence data according to simulation data of the digital twin model;

determine robot action sequence data optimized according to the simulation data of the digital twin model as a continuous action learned by the robot.