Electronic apparatus and controlling method thereof

1 . An electronic apparatus, comprising:

memory storing one or more instructions; and

at least one processor, comprising processing circuitry, coupled with the memory and configured to control the electronic apparatus,

wherein the one or more instructions, when executed by the at least one processor, causes the electronic apparatus to:

obtain a moving image,

identify a person and pose data of the person from a plurality of frames in the moving image,

obtain exercise pattern information corresponding to the plurality of frames using pose data of the identified person,

recognize an exercise motion of the identified person by inputting at least one frame of the moving image into at least one neural network model, and obtain exercise feature information corresponding to the recognized exercise motion,

identify a first frame interval from among the plurality of frames in the moving image, the first frame interval having a length determined, based on the exercise feature information and the exercise pattern information,

identify a second frame interval, different from the first frame interval, from among the plurality of frames in the moving image, the second frame interval having a specified length from a current frame,

compare first exercise pattern information for the first frame interval and second exercise pattern information for the second frame interval, and

provide information on an exercise motion corresponding to the second frame interval based on the comparing.

2 . The electronic apparatus of claim 1 , wherein

the memory stores one or more instructions that, when executed by at least one processor comprising processing circuitry, cause the electronic apparatus to:

input at least one frame of the moving image into a first neural network model configured to identify whether an exercise motion of a person is a symmetrical motion, and identify whether the exercise motion of the person is a symmetrical motion, and

input at least one frame of the moving image into a second neural network model configured to identify whether an exercise motion of a person is a hold motion, and identify whether the exercise motion of the person is a hold motion.

3 . The electronic apparatus of claim 2 , wherein:

the first neural network model is configured to be trained based on a learning moving image comprising an exercise motion of a person and information on whether the exercise motion of the person corresponding to the learning moving image is a symmetrical exercise motion, and

the second neural network model is configured to be trained based on a learning moving image comprising an exercise motion of a person and information on whether the exercise motion of the person corresponding to the learning moving image is a hold motion.

4 . The electronic apparatus of claim 1 , wherein

the memory stores one or more instructions that, when executed by at least one processor comprising processing circuitry, cause the electronic apparatus to:

identify, as the first frame interval, an interval from a start frame to a frame corresponding to a first time, based on an exercise motion of the person being a symmetrical motion and a hold motion.

5 . The electronic apparatus of claim 1 , wherein

the memory stores one or more instructions that, when executed by at least one processor comprising processing circuitry, cause the electronic apparatus to:

identify, based on an exercise motion of the person being a symmetrical motion and a non-hold motion, each exercise repetition interval from the plurality of frames in the moving image based on the exercise pattern information, and

identify, as the first frame interval, an interval from a start frame to a frame corresponding to an exercise repetition interval of a first number of times, based on the identified exercise repetition intervals.

6 . The electronic apparatus of claim 1 , wherein the memory stores one or more instructions that, when executed by at least one processor comprising processing circuitry, causes the electronic apparatus to:

identify, as the first frame interval, based on an exercise motion of the person being a non-symmetrical motion and a hold motion, an interval corresponding to an interval from a frame in which a first symmetrical exercise motion is first started to a first time and an interval from a frame in which a second symmetrical exercise motion is first started to a second time.

7 . The electronic apparatus of claim 1 , wherein

the memory stores one or more instructions that, when executed by at least one processor comprising processing circuitry, cause the electronic apparatus to:

identify, based on an exercise motion of the person being a non-symmetrical motion and a non-hold motion, each frame corresponding to a first symmetrical exercise motion and a second symmetrical exercise motion from the plurality of frames in the moving image based on the exercise pattern information, and

identify, as the first frame interval, a frame corresponding to the first symmetrical exercise motion of a first number of times and a frame corresponding to the second symmetrical exercise motion of a second number of times from a start frame.

8 . The electronic apparatus of claim 1 , wherein

the memory stores one or more instructions that, when executed by at least one processor comprising processing circuitry, cause the electronic apparatus to:

identify a physically depleted state based on a difference value of exercise pattern information corresponding to the first frame interval and exercise pattern information of the second frame interval being greater than or equal to a specified value, and provide, based on identifying the physical depleted state, information on an exercise motion of the person corresponding to the second frame interval.

9 . The electronic apparatus of claim 1 , wherein

the memory stores one or more instructions that, when executed by at least one processor comprising processing circuitry causes the electronic apparatus to:

obtain an original moving image, and

obtain, based on an exercise database (DB) storing a plurality of moving images that comprise a plurality of exercise motions, the moving image by identifying a frame corresponding to an exercise motion of the person from among a plurality of frames comprising in the original moving image.

10 . A method of controlling an electronic apparatus, the method comprising:

obtaining a moving image;

identifying a person and pose data of the person from a plurality of frames in the moving image;

obtaining exercise pattern information corresponding to the plurality of frames using pose data of the identified person;

recognizing an exercise motion of the identified person by inputting at least one frame of the moving image into at least one neural network model, and obtaining exercise feature information corresponding to the recognized exercise motion;

identifying a first frame interval from among the plurality of frames in the moving image, the first frame interval having a length determined, based on the exercise feature information and the exercise pattern information;

identifying a second frame interval, different from the first frame interval, from among the plurality of frames in the moving image, the second frame interval having a specified length from a current frame;

comparing first exercise pattern information for the first frame interval and second exercise pattern information for the second frame interval; and

providing information on an exercise motion corresponding to the second frame interval based on the comparing.

11 . The method of claim 10 , further comprising:

inputting at least one frame of the moving image into a first neural network model to identify whether an exercise motion of a person is a symmetrical motion, and identifying whether an exercise motion of the person is a symmetrical motion, and

inputting at least one frame of the moving image into a second neural network model to identify whether an exercise motion of a person is a hold motion, and identifying whether an exercise motion of the person is a hold motion.

12 . The method of claim 11 , wherein:

the first neural network model is configured to be trained based on a learning moving image comprising an exercise motion of a person and information on whether the exercise motion of the person corresponding to the learning moving image is a symmetrical motion, and

the second neural network model is configured to be trained based on a learning moving image comprising an exercise motion of a person and information on whether the exercise motion of the person corresponding to the learning moving image is a hold motion.

13 . The method of claim 10 , further comprising:

identifying, as the first frame interval, an interval from a start frame to a frame corresponding to a first time, based on an exercise motion of the person being a symmetrical motion and a hold motion.

14 . The method of claim 13 , further comprising:

identifying, based on an exercise motion of the person being a non-symmetrical motion and a non-hold motion, each frame corresponding to a first symmetrical motion and a second symmetrical motion from the plurality of frames in the moving image based on the exercise pattern information, and

identifying, as the first frame interval, a frame corresponding to the first symmetrical motion of a first number of times and a frame corresponding to the second symmetrical motion of a second number of times from a start frame.

15 . The method of claim 10 , further comprising:

identifying, based on an exercise motion of the person being a symmetrical exercise motion and a non-hold motion, each exercise repetition interval from the plurality of frames in the moving image based on exercise pattern information, and

identifying, as the first frame interval, an interval from a start frame to a frame corresponding to an exercise repetition interval of a first number of times, based on the identified exercise repetition intervals.

16 . The method of claim 10 , further comprising:

identifying, as the first frame interval, based on an exercise motion of the person being a non-symmetrical motion and a hold motion, an interval corresponding to an interval from a frame in which a first symmetrical exercise motion is first started to a first time and an interval from a frame in which a second symmetrical exercise motion is first started to a second time.

17 . The method of claim 10 , further comprising:

identifying a physically depleted state based on a difference value of exercise pattern information corresponding to the first frame interval and exercise pattern information of the second frame interval being greater than or equal to a specified value, and

providing, based on identifying the physical depleted state, information on an exercise motion of the person corresponding to the second frame interval.

18 . A non-transitory computer readable recording medium having recorded thereon a program that, when executed by at least one processor, causes an electronic apparatus to perform operations comprising:

obtaining a moving image;

identifying a person and pose data of the person from a plurality of frames in the moving image;

obtaining exercise pattern information corresponding to the plurality of frames using pose data of the identified person;

recognizing an exercise motion of the identified person by inputting at least one frame of the moving image into at least one neural network model, and obtaining exercise feature information corresponding to the recognized exercise motion;

identifying a first frame interval from among the plurality of frames in the moving image, the first frame interval having a length determined, based on the exercise feature information and the exercise pattern information;

identifying a second frame interval, different from the first frame interval, from among the plurality of frames in the moving image, the second frame interval having a specified length from a current frame;

compare first exercise pattern information for the first frame interval and second exercise pattern information for the second frame interval; and

providing information on an exercise motion corresponding to the second frame interval based on the comparing.