Granted Patent B2

US 12,734,404 · App. 17/768,019 · Granted Sep 15, 2026

Inventor: Yong Geun Lee (Seoul, KR)

Assignee: MOAIS, INC.

A63B24/0006A63B24/0003A63B69/3629G06T7/248G06T7/74G06V10/70G06V10/82G06V40/103G06V40/23A63B2024/0009A63B2024/0012A63B2102/32A63B2220/05A63B2220/10A63B2220/16A63B2220/807G06T2207/20084G06T2207/30196G06T2207/30221

Abstract

Claims (45)

1 . A method performed in a device for estimating information on a golf swing, the device comprising one or more processors and the method comprising the steps of:

by the one or more processors, light-weighting an artificial neural network model using depthwise convolution and pointwise convolution to obtain a light-weighted artificial neural network model;

by the one or more processors, when a photographed image of a user's golf swing is acquired, detecting at least one joint of the user from the photographed image using the light-weighted artificial neural network model;

by the one or more processors, estimating a posture of the user with reference to at least one of a type of the at least one joint of the user, a position of the at least one joint of the user, a distance between the at least one joint of the user and at least one other joint of the user, and an angle formed between the at least one joint of the user and at least one other joint of the user;

by the one or more processors, comparing the user's golf swing and a golf swing of a comparison target with reference to the posture of the user and at least one comparison point; and

by the one or more processors, estimating information on the user's golf swing on the basis of a result of the comparison,

wherein the at least one comparison point includes at least one of the position of the at least one joint, a position of a specific body part of the user estimated from the at least one joint, a reference line formed from the position of the at least one joint, and an angle formed from two or more reference lines,

wherein the at least one comparison point is established separately for each partial motion constituting each of the user's golf swing and the comparison target's golf swing, and established separately for each point of view with respect to the same partial motion,

wherein the user's golf swing and the comparison target's golf swing are constituted by a plurality of stages of partial motions, respectively, and similarity between the user's golf swing and the comparison target's golf swing is identified in consideration of the plurality of stages of partial motions, and

wherein the method further comprises:

extracting at least one frame corresponding to each stage of the plurality of stages of the user's golf swing from photographed images of the user's golf swing;

determining a comparison point for the extracted at least one frame corresponding to each stage of the plurality of stages of the user's golf swing, wherein the comparison point is varied depending on the stage to which the extracted at least one frame corresponds, and for each stage of the plurality stages of the user's golf swing, the comparison point is determined separately for each point of view including a front view and a side view of the user's golf swing; and

for each stage of the plurality of stages of the user's golf swing, comparing the comparison point for the extracted at least one frame corresponding to the stage of the user's golf swing to a target comparison point for a frame corresponding to a stage among the plurality of stages of the comparison target's golf swing corresponding to the stage of the user's golf swing, and

wherein the plurality of stages include an address stage, a takeaway stage, a back swing stage, a top-of-swing stage, a down swing stage, an impact stage, a follow-through stage, and a finish stage.

2 . The method of claim 1 , wherein in the step of comparing the user's golf swing and the comparison target's golf swing, the comparison target's golf swing is a golf swing of the user photographed at a different point of time from the user's golf swing.

3 . The method of claim 1 , wherein in the step of comparing the user's golf swing and the comparison target's golf swing, information on a golf club is estimated with reference to at least one of the type of the at least one joint of the user, the position of the at least one joint of the user, the distance between the at least one joint of the user and at least one other joint of the user, and the angle formed between the at least one joint of the user and at least one other joint of the user, and the user's golf swing and the comparison target's golf swing are compared with further reference to the estimated information on the golf club.

4 . The method of claim 1 , wherein storage of the acquired photographed image is started when the posture of the user satisfies a predetermined storage start condition, and the storage of the acquired photographed image is ended when the posture of the user satisfies a predetermined storage end condition.

5 . The method of claim 1 , wherein at least one of the detecting step, the step of estimating the user's posture, the step of comparing the user's golf swing and the comparison target's golf swing, and the step of estimating the information on the user's golf swing is performed according to a result of recognizing a voice from the user.

6 . The method of claim 1 , wherein a depth of each kernel in the depthwise convolution is reduced to 1 and each of a width and a height of each kernel in the pointwise convolution is reduced to 1.

7 . The method of claim 1 , wherein the detecting step comprises the steps of:

reconstructing the photographed image into a heat map image for each of the at least one joint of the user using the light-weighted artificial neural network model; and

detecting the at least one joint of the user by deriving the position of the at least one joint with reference to brightness values of points in each of the at least one heat map image, and

wherein a number of joints to be detected is limited such that at least one joint of importance not lower than a predetermined level is detected among joints of the user.

8 . The method of claim 1 , wherein when the stage of the plurality of stages of the user's golf swing is the address stage, a waist bend angle is determined as the comparison point.

9 . A non-transitory computer-readable recording medium having stored thereon a computer program that, when executed by a processor, causes the processor to perform the method of claim 1 .

10 . A device for estimating information on a golf swing, the device comprising one or more processors configured to:

light-weight an artificial neural network model using depthwise convolution and pointwise convolution to obtain a light-weighted artificial neural network model;

when a photographed image of a user's golf swing is acquired, detect at least one joint of the user from the photographed image using the light-weighted artificial neural network model;

estimate a posture of the user with reference to at least one of a type of the at least one joint of the user, a position of the at least one joint of the user, a distance between the at least one joint of the user and at least one other joint of the user, and an angle formed between the at least one joint of the user and at least one other joint of the user;

compare the user's golf swing and a golf swing of a comparison target with reference to the posture of the user and at least one comparison point; and

estimate information on the user's golf swing on the basis of a result of the comparison,

wherein the at least one comparison point includes at least one of the position of the at least one joint, a position of a specific body part of the user estimated from the at least one joint, a reference line formed from the position of the at least one joint, and an angle formed from two or more reference lines,

wherein the at least one comparison point is established separately for each partial motion constituting each of the user's golf swing and the comparison target's golf swing, and established separately for each point of view with respect to the same partial motion,

wherein the user's golf swing and the comparison target's golf swing are constituted by a plurality of stages of partial motions, respectively, and similarity between the user's golf swing and the comparison target's golf swing is identified in consideration of the plurality of stages of partial motions, and

wherein the one or more processors are further configured to:

extract at least one frame corresponding to each stage of the plurality of stages of the user's golf swing from photographed images of the user's golf swing;

determine a comparison point for the extracted at least one frame corresponding to each stage of the plurality of stages of the user's golf swing, wherein the comparison point is varied depending on the stage to which the extracted at least one frame corresponds, and for each stage of the plurality stages of the user's golf swing, the comparison point is determined separately for each point of view including a front view and a side view of the user's golf swing; and

for each stage of the plurality of stages of the user's golf swing, compare the comparison point for the extracted at least one frame corresponding to the stage of the user's golf swing to a target comparison point for a frame corresponding to a stage among the plurality of stages of the comparison target's golf swing corresponding to the stage of the user's golf swing, and

wherein the plurality of stages include an address stage, a takeaway stage, a back swing stage, a top-of-swing stage, a down swing stage, an impact stage, a follow-through stage, and a finish stage.

11 . The device of claim 10 , wherein a depth of each kernel in the depthwise convolution is reduced to 1 and each of a width and a height of each kernel in the pointwise convolution is reduced to 1.

12 . The device of claim 10 , wherein the one or more processors are configured to:

reconstruct the photographed image into a heat map image for each of the at least one joint of the user using the light-weighted artificial neural network model; and

detect the at least one joint of the user by deriving the position of the at least one joint with reference to brightness values of points in each of the at least one heat map image, and

wherein a number of joints to be detected is limited such that at least one joint of importance not lower than a predetermined level is detected among joints of the user.

13 . The device of claim 10 , wherein when the stage of the plurality of stages of the user's golf swing is the address stage, a waist bend angle is determined as the comparison point.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (19)