Dance game and tutorial
Presented herein are methods, apparatuses, programs, and systems for scoring a player performance comprising one or more poses in a dance-based video game. a performance 3D skeleton is received that indicates a pose of the player. A score is calculated by comparing a position, a timing, or both, associated with one or more joints of the performance 3D skeleton to a position, a timing, or both, associated with one or more joints of a target pose and altering one or more characteristics of the dance-based video game based on the score.
1. A method, executed on at least one processor, for scoring a player performance comprising one or more poses in a dance-based video game based on input received via a sensor, the method comprising:
receiving, by the at least one processor, a performance 3D skeleton indicating a pose of the player;
calculating, using the at least one processor, a score by comparing a position, a timing, or both, associated with one or more joints of the performance 3D skeleton to a position, a timing, or both, associated with one or more joints of a target pose; and
altering, using the at least one processor, one or more characteristics of the dance-based video game based on the score.
2. The method of claim 1 further comprising generating, using the at least one processor, the target pose based on motion capture data.
3. The method of claim 1 wherein the position associated with the one or more joints of the 3D skeleton and the position associated with the one or more joints of the target pose are based on a normalization of the spatial position of the one or more joints of the 3D skeleton and the one or more joints of the target pose, respectively.
4. The method of claim 3 wherein the normalizing the spatial position of a joint comprises computing a unit vector reflecting an offset of the joint relative to an origin joint.
5. The method of claim 4 wherein the normalizing the spatial position of the joint comprises:
defining a coordinate system originated at the origin joint; and
translating the unit vector into the coordinate system.
6. The method of claim 5 wherein the origin joint is one of left shoulder, right shoulder, left hip, or right hip.
7. The method of claim 3 wherein the normalizing the spatial position of a joint comprises:
computing, using the at least one processor, a first vector reflecting the offset of the joint relative to an origin joint;
computing, using the at least one processor, a second vector reflecting the offset of an intermediate joint relative to the origin joint;
computing, using the at least one processor, a third vector reflecting the offset of the joint relative to the intermediate joint; and
dividing, using the at least one processor, the first vector by the sum of the magnitudes of the second and third unit vectors.
8. A method, executed on at least one processor, for evaluating a player performance based on input received via a sensor, the method comprising:
receiving, by the at least one processor, a performance 3D skeleton indicating a portion of the player performance;
providing, by the at least one processor, a target 3D skeleton indicating a portion of a target performance;
defining, using the at least one processor, a per joint error function;
calculating, using the at least one processor, an error using the per joint error function based on the performance 3D skeleton and the target 3D skeleton; and
producing, using the at least one processor, an audio or visual indication of the error.
9. A method, executed on at least one processor, for scoring a player performance comprising one or more poses in a dance-based video game based on input received via a sensor, the method comprising:
providing, by the at least one processor, a target 3D skeleton indicating a target pose;
receiving, by the at least one processor, a plurality of 3D performance skeletons indicating player poses;
generating, using the at least one processor, an overall score for a particular 3D performance skeleton by comparing a position associated with one or more joints of the 3D performance skeleton to a corresponding position associated with one or more joints of the target 3D skeleton;
repeating, using the at least one processor, the score generating step for each of the plurality of 3D performance skeletons that fall within a predetermined temporal range to generate a plurality of overall scores; and
producing, on a display in communication with the at least one processor, an audio or visual indication based on one or more of the plurality of overall scores.
10. The method of claim 9 further comprising:
providing, by the at least one processor, a second target 3D skeleton indicating a second target pose;
receiving, by the at least one processor, a second plurality of 3D performance skeletons indicating player poses;
generating, using the at least one processor, an overall score for a second particular 3D performance skeleton by comparing a position associated with one or more joints of the second 3D performance skeleton to a corresponding one or more joints of the second target 3D skeleton;
repeating, using the at least one processor, the score generating step for each of the second plurality of 3D performance skeletons that fall within a second predetermined temporal range to generate a second plurality of overall scores; and
producing, on the display in communication with the at least one processor, a second audio or visual indication based on one or more of the first and second plurality of overall scores.
11. The method of claim 9 wherein the overall score comprises a positional score and a timing-based score.
12. The method of claim 11 wherein the target pose is associated with a target performance timing, and the timing-based score varies inversely as the difference between the target performance timing and a timing of performance of the player pose.
13. The method of claim 11 wherein the timing-based score is a first constant if the difference is less than a first predetermined threshold, varies inversely as the difference if the difference is between the first predetermined threshold and a second predetermined threshold, and is a second constant if the difference is greater than the second predetermined threshold.
14. The method of claim 11 wherein the positional score comprises one or more per joint scores.
15. The method of claim 14 wherein the contribution of each per-joint score to the overall score is weighted.
16. The method of claim 11 wherein the positional score comprises one or more body part scores, each body part score comprising one or more per-joint scores.
17. The method of claim 16 wherein the contribution of each body part score to the overall score is weighted.
18. The method of claim 9 wherein the overall score comprises a displacement score.
19. The method of claim 9 wherein the overall score comprises a velocity score.
20. The method of claim 9 wherein the overall score comprises an acceleration score.
21. The method of claim 9 wherein the comparing step comprises calculating, using the at least one processor, a Euclidean distance between the position associated with the one or more joints of the 3D performance skeleton and the corresponding position associated with the one or more joints of the target 3D skeleton to generate a per-joint score.
22. The method of claim 21 wherein the per-joint score vanes inversely as the Euclidean distance.
23. The method of claim 21 wherein the per-joint score is a first constant if the Euclidean distance is less than a first predetermined threshold, varies inversely as the Euclidean distance if the Euclidean distance is between the first predetermined threshold and a second predetermined threshold, and is a second constant if the Euclidean distance is greater than the second predetermined threshold.
24. The method of claim 21 wherein calculating, using the at least one processor, the Euclidean distance comprises weighting the contributions of the x, y, and z axes to the Euclidean distance.
25. The method of claim 24 wherein the orientations of the x, y, and z axes are relative to each body zone.
26. The method of claim 25 wherein the x, y, and z axes of an arm body zone originate at a shoulder connected to the arm and are based on a first vector from the shoulder to an opposite shoulder, a second vector in the direction of gravity, and a third vector that is a cross product of the first and second vectors.
27. The method of claim 25 wherein the x, y, and z axes of a leg body zone originate at a hip connected to the leg and are based on a first vector from the hip to an opposite hip, a second vector in the direction of gravity, and a third vector that is a cross product of the first and second vectors.
28. The method of claim 24 wherein the orientations of the x, y, and z axes are relative to the target 3D skeleton.
29. The method of claim 28 wherein the x, y, and z axes are rotated based on an angle between a vector normal to a plane of the target 3D skeleton and a vector normal to a view plane.
30. The method of claim 9 wherein the audio or visual indication comprises a visual indication of a body part score.
31. The method of claim 9 wherein the audio or visual indication comprises an audible indication of a body part score.
32. The method of claim 9 wherein the audio or visual indication comprises one more graphics, the number of graphics based on the one or more overall scores.
33. The method of claim 9 wherein the audio or visual indication comprises a simulated crowd and one or more characteristics of the simulated crowd varies based on the one or more of the plurality of overall scores.
34. The method of claim 33 wherein the one or more characteristics of the simulated crowd comprises the number of people in the crowd.
35. The method of claim 9 wherein the audio or visual indication comprises a simulated environment and one or more characteristics of the simulated environment varies based on the one or more of the plurality of overall scores.
36. The method of claim 9 wherein the target pose is selected based on its importance to the target performance.
37. A method, executed on at least one processor, for scoring a player performance comprising one or more poses in a dance-based video game based on input received via a sensor, the method comprising:
receiving, by the at least one processor, a performance 3D skeleton indicating a pose of the player;
calculating, using the at least one processor, a score by comparing a measurement of one or more reference points of the performance 3D skeleton to a measurement of one or more reference points of a target pose; and
altering, using the at least one processor, one or more characteristics of the dance-based video game based on the score.
38. The method of claim 37 wherein the measurement is a displacement.
39. The method of claim 37 wherein the measurement is a velocity.
40. The method of claim 37 wherein the measurement is an acceleration.
41. The method of claim 37 wherein the one or more reference points are one or more joints.
42. The method of claim 37 wherein the one or more reference points are one or more bones.
43. The method of claim 37 wherein the one or more reference points are derived from one or more joints.
44. The method of claim 37 wherein the one or more reference points are derived from one or more bones.
45. A computer program product, tangibly embodied in a non-transitory computer readable storage medium, for scoring a player performance comprising one or more poses in a dance-based video game based on input received via a sensor, the computer program product including instructions being operable to cause a data processing apparatus to:
receive a performance 3D skeleton indicating a pose of the player;
calculate a score by comparing a position, a timing, or both, associated with one or more joints of the performance 3D skeleton to a position, a timing, or both, associated with one or more joints of a target pose; and
alter one or more characteristics of the dance-based video game based on the score.
46. A computer program product, tangibly embodied in a non-transitory computer readable storage medium, for evaluating a player performance based on input received via a sensor, the computer program product including instructions being operable to cause a data processing apparatus to:
receive a performance 3D skeleton indicating a portion of the player performance;
provide a target 3D skeleton indicating a portion of a target performance;
define a per joint error function;
calculate an error using the per joint error function based on the performance 3D skeleton and the target 3D skeleton; and
produce, with the game platform, an audio or visual indication of the error.
47. A computer program product, tangibly embodied in a non-transitory computer readable storage medium, for scoring a player performance comprising one or more poses in a dance-based video game based on input received via a sensor, the computer program product including instructions being operable to cause a data processing apparatus to:
provide a target 3D skeleton indicating a target pose;
receive a plurality of 3D performance skeletons indicating player poses;
generate an overall score for a particular 3D performance skeleton by comparing a position associated with one or more joints of the 3D performance skeleton to a corresponding position associated with one or more joints of the target 3D skeleton;
repeat the score generating step for each of the plurality of 3D performance skeletons that fall within a predetermined temporal range to generate a plurality of overall scores; and
produce, on a display in communication with the game platform, an audio or visual indication based on one or more of the plurality of overall scores.
48. A computer program product, tangibly embodied in a non-transitory computer readable storage medium, for scoring a player performance comprising one or more poses in a dance-based video game based on input received via a sensor, the computer program product including instructions being operable to cause a data processing apparatus to:
receive a performance 3D skeleton indicating a pose of the player;
calculate a score by comparing a measurement of one or more reference points of the performance 3D skeleton to a measurement of one or more reference points of a target pose; and
alter one or more characteristics of the dance-based video game based on the score.
49. A system for scoring a player performance comprising one or more poses in a dance-based video game, comprising:
a sensor to receive an input; and
a game platform configured to:
receive a performance 3D skeleton indicating a pose of the player;
calculate a score by comparing a position, a timing, or both, associated with one or more joints of the performance 3D skeleton to a position, a timing, or both, associated with one or more joints of a target pose; and
alter one or more characteristics of the dance-based video game based on the score.
50. A system for evaluating a player performance, the system comprising:
a sensor to receive an input; and
a game platform configured to:
receive a performance 3D skeleton indicating a portion of the player performance;
provide a target 3D skeleton indicating a portion of a target performance;
define a per joint error function;
calculate an error using the per joint error function based on the performance 3D skeleton and the target 3D skeleton; and
produce an audio or visual indication of the error.
51. A system for scoring a player performance comprising one or more poses, the system comprising:
a display;
a sensor to receive an input; and
a game platform configured to:
provide a target 3D skeleton indicating a target pose;
receive a plurality of 3D performance skeletons indicating player poses;
generate an overall score for a particular 3D performance skeleton by comparing a position associated with one or more joints of the 3D performance skeleton to a corresponding position associated with one or more joints of the target 3D skeleton;
repeat the score generating step for each of the plurality of 3D performance skeletons that fall within a predetermined temporal range to generate a plurality of overall scores; and
produce, on the display, an audio or visual indication based on one or more of the plurality of overall scores.
52. A system for scoring a player performance comprising one or more poses in a dance-based video game, the system comprising:
a sensor to receive an input; and
a-game platform configured to:
receive a performance 3D skeleton indicating a pose of the player;
calculate a score by comparing a measurement of one or more reference points of the performance 3D skeleton to a measurement of one or more reference points of a target pose; and
alter one or more characteristics of the dance-based video game based on the score.