Granted Patent B2

US 12,383,789 · App. 18/491,771 · Granted Aug 12, 2025

Inventors: Keng Fai Lee (Cupertino, CA); Qi Zhang (Tseung Kwan O, HK); Man Hon Chan (Kowloon, HK); On Loy Sung (Lai Chi Kok, HK); Jorge Fino (San Jose, CA)

Assignee: NEX Team Inc.

A63B24/0006A63B24/0021A63B24/0062A63B71/0616A63B71/0622G06N3/08G06T7/20G06T7/70G06V20/46G06V40/10G06V40/23A63B2024/0009A63B2024/0056A63B2024/0065A63B2071/0677G06T2207/20081G06T2207/20084G06T2207/30196G06T2207/30221

Abstract

Claims (48)

1. A computer-implemented method executable by a hardware processor for video processing, comprising:

receiving an input video of one or more persons from a camera;

detecting multiple human poses in a given frame of the input video using an artificial intelligence (AI)-based computer vision module, wherein each of the human poses comprises linked key points, wherein the key points represent human joints, and wherein the key points are identified by the AI-based computer vision module;

selecting a proper pose of a user from among the multiple human poses in the given frame of the input video, to generate a sequence of proper poses across multiple frames;

determining an activity performed by the user, based on one or more of the selected proper poses;

computing frame-to-frame changes in coordinates of the one or more key points in the sequence of proper poses across multiple frames;

computing a function of the changes in the coordinates of the one or more key points in the sequence of proper poses across multiple frames;

counting a given user movement as a repetitive motion of the activity based on the function, wherein the function is dependent on the activity performed by the user;

performing a checking process on the given user movement's metrics, based on a limb movement of the user, to invalidate the given user movement based on one or more criteria;

computing a plurality of statistics about the activity based on the counting and the checking process, wherein the plurality of statistics comprises a cadence of the repetitive motion; and

displaying on a user device, the plurality of the statistics about the activity.

2. The computer-implemented method of claim 1 , wherein the activity is selected from the group consisting of running, jogging, walking, jumping, performing jumping jacks, squatting, and dribbling.

3. The computer-implemented method of claim 1 , wherein the repetitive motion is selected from the group consisting of steps, jumps, squats, and dribbles.

4. The computer-implemented method of claim 1 , wherein the selecting the proper pose comprises:

selecting a centered pose of the multiple human poses in the given frame as the proper pose.

5. The computer-implemented method of claim 1 , wherein the selecting the proper pose comprises:

selecting the proper pose utilizing a human tracking algorithm.

6. The computer-implemented method of claim 1 , wherein the function of the changes in the coordinates is selected from the group consisting of a mean, a median, and a single delta value selection.

7. The computer-implemented method of claim 1 , wherein the function of the changes in the coordinates is a mean delta value, and wherein the method further comprises:

counting the given user movement as the repetitive motion when the mean delta value changes in a predetermined pattern.

8. The computer-implemented method of claim 1 , further comprising:

applying a smoothing function on the coordinates of the one or more key points.

9. The computer-implemented method of claim 1 , further comprising:

excluding the given user movement based on an associated rising period being more than a first threshold; and

excluding the given user movement based on an associated rising amplitude being smaller than a second threshold.

10. The computer-implemented method of claim 1 , further comprising:

adjusting the checking process based on a parameter.

11. The computer-implemented method of claim 1 , further comprising:

presenting one or more gamification elements based on at least one of the plurality of statistics.

12. The computer-implemented method of claim 1 , wherein the input video is captured using the camera selected from the group consisting of a mobile device camera and a portable camera device.

13. A non-transitory storage medium storing program code for video processing, the program code executable by a hardware processor, the program code when executed by the hardware processor causes the hardware processor to:

receive an input video of one or more persons from a camera;

detect multiple human poses in a given frame of the input video using an artificial intelligence (AI) based computer vision module, wherein each of the human poses comprises linked key points, wherein the key points represent human joints, and wherein the key points are identified by the AI-based computer vision module;

select a proper pose of a user from among the multiple human poses in the given frame of the input video, to generate a sequence of proper poses across multiple frames;

determine an activity performed by the user, based on one or more of the selected proper poses;

compute frame-to-frame changes in coordinates of the one or more key points in the sequence of proper poses across multiple frames;

compute a function of the changes in the coordinates of the one or more key points in the sequence of proper poses across multiple frames;

count a given user movement as a repetitive motion of the activity based on the function, wherein the function is dependent on the activity performed by the user;

perform a checking process on the given user movement's metrics, based on a limb movement of the user, to invalidate the given user movement based on one or more criteria;

compute a plurality of statistics about the activity based on the counting and the checking process, wherein the plurality of statistics comprises a cadence of the repetitive motion; and

display on a user device, the plurality of the statistics about the activity.

14. The non-transitory storage medium of claim 13 , wherein the activity is selected from the group consisting of running, jogging, walking, jumping, performing jumping jacks, squatting, and dribbling.

15. The non-transitory storage medium of claim 13 , wherein the program code to select the proper pose comprises program code to:

select a centered pose of the multiple human poses in the given frame as the proper pose.

16. The non-transitory storage medium of claim 13 , wherein the program code to select the proper pose comprises program code to:

select the proper pose utilizing a human tracking algorithm.

17. The non-transitory storage medium of claim 13 , wherein the function of the changes in the coordinates is a mean delta value, and wherein the program code further causes the hardware processor to:

count the given user movement as the repetitive motion when the mean delta value changes in a predetermined pattern.

Assignments (2)

Continuity (3)

References Cited (24)