Movement based recognition and evaluation
This disclosure relates generally to analysis of movement that is electronically captured and recognition of which movement was performed for purposes of evaluation. This type of motion capture, analysis and evaluation can have applicability across a number of fields, such as healthcare, entertainment, communication, and manufacturing. The recognition can occur once movement information collected from sensors is inputted into the system. The system can then analyze the movement information to determine what the movement is. In some games, such as sports games, movement recognition can provide feedback to a user about what they are doing. In other games, such as dance games, movement recognition can allow for users to dance a move from a set of moves providing for a more entertaining game.
1. A method executed on a game console comprising:
instructing a display to display a set of dance moves that are available for performance during a song in a dance game, wherein the set of dance moves is related to a target performance that comprises a plurality of frames, each frame being associated with a target performance skeleton that is normalized by deriving one or more target unit vectors, each of which reflects a target position of one interest point of the target performance skeleton relative to another interest point of the target performance skeleton,
wherein a subset of the plurality of frames comprises at least one key frame, and
wherein the target performance skeleton is represented, at least in part by, the one or more target unit vectors;
receiving, at the dance game executing on the game console, an input performance by a player of the dance game, wherein the input performance is indicative of a performance of the player, and wherein the input performance comprises an input performance skeleton having a first interest point and a second interest point;
normalizing, at the dance game executing on the game console, the input performance skeleton of the input performance by deriving an input unit vector reflecting an input position of the first interest point of the input performance skeleton relative to the second interest point of the input performance skeleton, wherein the input performance skeleton is represented, at least in part by, the input unit vector; and
determining a degree of matching between the input performance and the at least one key frame of the target performance based, in part, on a similarity of the normalized target performance skeleton and the normalized input performance skeleton.
2. The method of claim 1 , wherein:
the input performance comprises a plurality of frames, each frame being associated with an input performance skeleton,
each input performance skeleton is normalized by deriving an input unit vector reflecting an input position of a first interest point of the corresponding input performance skeleton relative to a second interest point of the corresponding input performance skeleton, and
the degree of matching is determined to be zero when none of the frames of the input performance within a predetermined time window matches the at least one key frame.
3. The method of claim 1 , wherein the at least one key frame provides differing weights to different aspects of the position and movement information of the target performance in determining the degree of matching.
4. The method of claim 1 , further comprising:
assigning a score based on the degree of matching between the input performance and each of a plurality of target performances; and
recognizing a match between the input performance and one of the plurality of target performances when the score exceeds a predetermined dance move performance threshold.
5. The method of claim 1 , further comprising:
instructing the display to display a plurality of target performances as icons that are animated; and
offsetting a beginning of the animation relative to beats of the song.
6. The method of claim 1 , further comprising:
receiving sensory information that includes a plurality of input performances; and
evaluating the plurality of input performances against a plurality of target performances to determine matches between the plurality of input performances and each of the plurality of target performances.
7. The method of claim 1 , further comprising:
selecting a target performance based on the degree of matching between the input performance and each of a plurality of target performances, wherein a higher degree of matching indicates a closer match; and
adding the selected target performance to a dance routine that is being created.
8. The method of claim 1 , wherein normalizing the input performance skeleton comprises determining a vector associated with an input position of a joint relative to a reference joint in the input performance skeleton.
9. The method of claim 1 , wherein normalizing the input performance skeleton comprises normalizing a first joint relative to a first reference joint in the input performance skeleton and normalizing a second joint relative to a second reference joint in the input performance skeleton.
10. The method of claim 3 , wherein the weights correspond to different bone groups of the target performance skeleton to emphasize a particular motion in determining the degree of matching.
11. The method of claim 1 , wherein the at least one key frame provides differing weights to different dimensions of movements associated with the input performance in determining the degree of matching.
12. The method of claim 1 , wherein the at least one key frame comprises a time offset relative to a beginning of the target performance.
13. The method of claim 1 , wherein determining the degree of matching between the input performance and the at least one key frame of the target performance comprises determining the degree of matching between the input performance and the at least one key frame of the target performance based solely on the similarity of the at least one key frame and frames of the normalized input performance within a predetermined time window.
14. A non-transitory computer readable medium that comprises computer readable instructions that when executed perform steps comprising:
instruct a display to display a set of dance moves that are available for performance during a song in a dance game, wherein the set of dance moves is related to a target performance that comprises a plurality of frames, each frame being associated with a target performance skeleton that is normalized by deriving one or more target unit vectors, each of which reflects a target position of one interest point of the target performance skeleton relative to another interest point of the target performance skeleton,
wherein a subset of the plurality of frames comprises at least one key frame, and
wherein the target performance skeleton is represented, at least in part by, the one or more target unit vectors;
receive an input performance from a player of the dance game, wherein the input performance is indicative of a performance of the player, and wherein the input performance comprises an input performance skeleton having a first interest point and a second interest point;
normalize the input performance skeleton of the input performance by deriving an input unit vector reflecting an input position of the first interest point of the input performance skeleton relative to the second interest point of the input performance skeleton, wherein the input performance skeleton is represented, at least in part by, the input unit vector; and
determine a degree of matching between the input performance and the at least one key frame of the target performance based, in part, on a similarity of the normalized target performance skeleton and the normalized input performance skeleton.
15. The computer readable medium of claim 14 , wherein:
the input performance comprises a plurality of frames, each frame being associated with an input performance skeleton,
each input performance skeleton is normalized by deriving an input unit vector reflecting an input position of a first interest point of the corresponding input performance skeleton relative to a second interest point of the corresponding input performance skeleton, and
the degree of matching is determined to be zero when none of the frames of the input performance within a predetermined time window matches the at least one key frame.
16. The computer readable medium of claim 14 , wherein the at least one key frame provides differing weights to different aspects of the position and movement information of the target performance in determining the degree of matching.
17. The computer readable medium of claim 14 , further comprising computer readable instructions that, when executed, perform steps of:
assigning a score based on the degree of matching between the input performance and each of a plurality of target performances; and
recognizing a match between the input performance and one of the plurality of target performances when the score exceeds a predetermined dance move performance threshold.
18. The computer readable medium of claim 14 , further comprising computer readable instructions that, when executed, perform steps of:
receiving sensory information that includes a plurality of input performances; and
evaluating the plurality of input performances against a plurality of target performances to determine matches between the plurality of input performances and each of the plurality of target performances.
19. The non-transitory computer readable medium of claim 14 , wherein the at least one key frame provides differing weights to different dimensions of movements associated with the input performance in determining the degree of matching.
20. The non-transitory computer readable medium of claim 14 , wherein the at least one key frame comprises a time offset relative to a beginning of the target performance.
21. A game platform system configured to receive movement sensor information including an input performance by a player of a dance game, wherein the input performance is indicative of a performance of the player, wherein the input performance comprises an input performance skeleton having a first interest point and a second interest point, and the game platform system includes a processor configured to read instructions from memory to:
instruct a display to display a set of dance moves that are available for performance during a song in the dance game, wherein the set of dance moves is related to a target performance that comprises a plurality of frames, each frame being associated with a target performance skeleton that is normalized by deriving one or more target unit vectors, each of which reflects a target position of one interest point of the target performance skeleton relative to another interest point of the target performance skeleton,
wherein a subset of the plurality of frames comprises at least one key frame, and
wherein the target performance skeleton is represented, at least in part by, the one or more target unit vectors;
normalize the input performance skeleton of the input performance by deriving an input unit vector reflecting an input position of the first interest point of the input performance skeleton relative to the second interest point of the input performance skeleton, wherein the input performance skeleton is represented, at least in part by, the input unit vector; and
determine a degree of matching between the input performance and the at least one key frame of the target performance based, in part, on a similarity of the normalized target performance skeleton and the normalized input performance skeleton.
22. The game platform of claim 21 , wherein:
the input performance comprises a plurality of frames, each frame being associated with an input performance skeleton,
each input performance skeleton is normalized by deriving an input unit vector reflecting an input position of a first interest point of the corresponding input performance skeleton relative to a second interest point of the corresponding input performance skeleton, and
the degree of matching is determined to be zero when none of the frames of the input performance within a predetermined time window matches the at least one key frame.
23. The game platform system of claim 21 , wherein the at least one key frame provides differing weights to different dimensions of movements associated with the input performance in determining the degree of matching.
24. The game platform system of claim 21 , wherein the at least one key frame comprises a time offset relative to a beginning of the target performance.