Method of scoring a move of a user and system thereof

1. A computerized method for scoring performance of a move of a trainee in a trainee frame input in relation to a move of a trainer in a trainer video, wherein the trainer move includes at least one trainer keyframe, the method comprising:

obtaining a trainee frame input comprising a trainee move, the trainee move comprising a plurality of unnormalized trainee frames;

based on the at least one trainer keyframe, processing the plurality of unnormalized trainee frames to provide a move performance score, indicative of the performance of the trainee move in relation to the trainer move, with respect to at least one aspect of the performance, the processing comprising:

selecting for the at least one trainer keyframe, according to a selection criterion, at least one unnormalized trainee frame of the plurality of trainee frames that correspond to the at least one trainer keyframe, giving rise to at least one candidate;

based on the at least one trainer keyframe and the at least one candidate, calculating at least one aspect score of the at least one aspect;

transforming the at least one calculated aspect score, giving rise to the move performance score; and

based on the move performance score, providing feedback to the trainee;

whereby, the provided feedback facilitates the trainee to improve performance of a future move with respect to the trainer move.

2. The computerized method of claim 1 , wherein the selection criterion is a time criterion, and wherein selecting the at least one trainee frame comprises selecting at least one candidate in a time window in the trainee frame input that is around a time point of the at least one trainer keyframe in the trainer video.

3. The computerized method of claim 1 , wherein the at least one aspect is a similarity aspect, and wherein the at least one trainer keyframe includes body parts of the trainer and the at least one candidate includes body parts of the trainee, and wherein calculating the similarity aspect score further comprises:

computing at least one angular difference between at least one body part included in the at least one trainer keyframe and at least one corresponding body part included in the at least one candidate; and

calculating the similarity aspect score, based on the computed at least one angular difference.

4. The computerized method of claim 3 , wherein computing the angular differences comprises computing at least one parameter selected from a group of parameters comprising: absolute angular difference, cosine distance, and a learned distance.

5. The computerized method of claim 3 , wherein the at least one trainer keyframe is associated with a predefined keyframe format, the method further comprising:

pre-processing the at least one candidate in accordance with the predefined keyframe format, giving rise to at least one formatted candidate, such that the at least one formatted candidate is formatted according to the predefined format; and

computing the angular differences between the body parts in the at least one trainer keyframe and body parts in the at least one formatted candidate.

6. The computerized method of claim 3 , wherein at least one body part of the at least one body part of the trainer is associated with a respective weight, and wherein the method further comprises:

computing the angular difference between the at least one body part and at least one corresponding body part of the trainee; and

associating the computed angular difference with the respective weight; and

calculating the similarity aspect score, based on the computed angular difference, according to the associated respective weight.

7. The computerized method of claim 1 , wherein the trainer move includes at least two trainer keyframes, wherein the at least one aspect is a timing aspect pertaining to the at least two trainer keyframes, and wherein prior to calculating the timing aspect score, the method further comprises:

obtaining, for each of the at least two candidates, a respective matching score, the matching score being indicative of a likelihood of match between a candidate and a trainer keyframe; and

wherein calculating the timing aspect score further comprises:

based on a trainer time interval in the trainer video, the trainer time interval includes the at least two trainer keyframes, determining a corresponding trainee time interval in the trainee frame input, the corresponding trainee time interval including at least two successive candidates having respective matching scores, and

calculating a timing score for the at least two successive candidates, with respect to one or more timing parameters;

wherein transforming the calculated timing score further comprises fusing the matching scores and the calculated respective timing score of the candidates, giving rise to the move performance score.

8. The computerized method of claim 7 , wherein the timing parameter is an out-of-sync parameter pertaining to an order of appearance of the at least two trainee frames in the sequence when compared to an order of appearance of the at least two trainer keyframes, and wherein calculating the respective timing score further comprises:

determining, for at least two trainee frames of the sequence of keyframes, an out-of-sync parameter score, and

calculating the timing score of each respective trainee frame based on at least the determined out-of-sync parameter scores.

9. The computerized method of claim 7 , wherein the timing parameter is a time offset parameter, wherein a time offset that pertains to an offset between a time that a trainer keyframe appears in the trainer video, and a time that the candidate appears in the trainee frame input, and wherein calculating the respective timing score further comprises:

determining for a trainer keyframe and a corresponding candidate a time offset; and

calculating the timing score based on the determined time offset scores.

10. The computerized method of claim 7 , wherein the at least one matching score is a similarity aspect score.

11. The computerized method of claim 7 , wherein the at least one matching score is a motion dynamics score.

12. The computerized method of claim 7 , wherein calculating the timing score is based on at least two timing parameters, wherein at least one of the timing parameters is associated with a weight, and wherein calculating the timing score further comprises calculating the timing score based on the associated weight.

13. The computerized method of claim 7 , prior to fusing the matching score and the timing score, the method further comprises:

with respect to at least one candidate of the at least two successive candidates, aggregating the obtained matching score and the calculated timing score of the candidates to provide an aggregated optimality score;

for at least one trainer keyframe, selecting a matching trainee frame by selecting one candidate having a highest aggregated optimality score; and

fusing the matching score and the timing score of the matching trainee frames, giving rise to a move performance score.

14. The computerized method of claim 1 , wherein the trainer move includes at least two trainer keyframes, and wherein at least one aspect being a motion dynamics aspect, and wherein:

based on a trainer time interval included in the trainer video, the trainer time interval includes the at least two trainer keyframes, determining a corresponding trainee time interval in the trainee frame input, the determined corresponding trainee time interval including at least two trainee frames;

based on at least one motion feature extracted from the at least two trainer keyframes included in the trainer time interval, determining a motion dynamics score for the at least two trainee frames;

wherein the at least one motion feature is indicative of movement transformation between two keyframes.

15. The computerized method of claim 14 , wherein at least one trainer keyframe of the at least two trainer keyframes is associated with a respective matching trainee frame, and wherein the determined corresponding trainee time interval includes at least one matching trainee frame to the at least two trainer keyframes.

16. The computerized method of claim 14 ,

wherein each of the trainer and corresponding trainee time intervals is associated with a window size, wherein the window size associated with the corresponding trainee time interval is different to the window size associated with the trainer time interval;

calculating a difference between the window sizes of the trainer and corresponding trainee time intervals;

normalizing the trainee frames included in the trainee time interval in accordance with the calculated difference; and

determining the at least one motion feature in the normalized trainee frames.

17. The computerized method of claim 14 , wherein the at least one motion feature is selected from a group of features comprising: peak motion feature, velocity motion feature, pose motion feature, relative change motion feature, and turn motion feature.

18. The computerized method of claim 1 , wherein processing the plurality of trainee frames to provide the move performance score is done with respect to at least first and second aspects, and wherein transforming the at least first and second calculated aspect scores further comprises:

for at least the first calculated aspect score of the first aspect, determining a respective transformation function, the transformation function being determined based on one or more conditions pertaining at least to the second calculated aspect score of the second aspect; and

fusing the at least first and second aspects scores, based on the respective transformation functions, giving rise to the move performance score.

19. A system for scoring performance of a move of a trainee in a trainee frame input in relation to a move of a trainer in a trainer video, wherein the trainer move includes at least one trainer keyframe, by a processor and memory circuitry (PMC), the processor being configured to:

obtain a trainee frame input comprising a trainee move, the trainee move comprising a plurality of unnormalized trainee frames;

based on the at least one trainer keyframe, process the plurality of unnormalized trainee frames to provide a move performance score, indicative of the performance of the trainee move in relation to the trainer move, with respect to at least one aspect of the performance, the process comprising:

selecting for the at least one trainer keyframe, according to a selection criterion, at least one unnormalized trainee frame of the plurality of trainee frames that correspond to the at least one trainer keyframe, giving rise to at least one candidate;

based on the at least one trainer keyframe and the at least one candidate, calculate at least one aspect score of the at least one aspect; and

transform the at least one calculated aspect score, giving rise to the move performance score; and

based on the move performance score, provide feedback to the trainee;

whereby, the provided feedback facilitates the trainee to improve performance of a future move with respect to the trainer move.

20. A non-transitory computer readable storage medium tangibly embodying a program of instructions that, when executed by a computer, cause the computer to perform a method for scoring performance of a move of a trainee in a trainee frame input in relation to a move of a trainer in a trainer video, wherein the trainer move includes at least two trainer keyframes, the method comprising:

obtaining a trainee frame input comprising a trainee move, the trainee move comprising a plurality of unnormalized trainee frames;

based on the at least one trainer keyframe, processing the plurality of unnormalized trainee frames to provide a move performance score, indicative of the performance of the trainee move in relation to the trainer move, with respect to at least one aspect of the performance, the processing comprising:

selecting for the at least one trainer keyframe, according to a selection criterion, at least one unnormalized trainee frame of the plurality of trainee frames that correspond to the at least one trainer keyframe, giving rise to at least one candidate;

based on the at least one trainer keyframe and the at least one candidate, calculating at least one aspect score of the at least one aspect; and

transforming the at least one calculated aspect score, giving rise to the move performance score; and

based on the move performance score, providing feedback to the trainee;

whereby the provided feedback facilitates the trainee to improve performance of a future move with respect to the trainer move.