System and method for automatic recognition of user motion

1. A system for automatically recognizing user motion, the system comprising:

an input unit configured to receive three-dimensional (3D) measurement data that is acquired by a low-cost depth sensor;

a memory which stores a program for performing automatic motion recognition on 3D user motion using 3D high-quality image data and a deep learning model; and

a processor configured to execute the program,

wherein the processor converts 3D low-quality depth data into the 3D high-quality image data, the 3D low-quality depth data being obtained from the 3D measurement data, and

wherein the processor, in order to convert the 3D low-quality depth data of a current time into the 3D high-quality image data of the current time, synthesizes the 3D low-quality depth data of the current time with 3D high-quality intermediate data generated at an immediately previous time to form 3D high-quality intermediate data that is required to generate the 3D high-quality image data of the current time.

2. The system of claim 1 , wherein the processor defines the deep learning model using 2D image data acquired by a camera and 2D user motion data.

3. The system of claim 1 , wherein the processor converts the 3D measurement data received by the input unit into the 3D low-quality depth data using a maximum measurable depth value that is a fixed value for the low-cost depth sensor.

4. The system of claim 1 , wherein the processor applies the deep learning model to a 2D image domain of the 3D high-quality image data to perform the automatic motion recognition.

5. The system of claim 1 , wherein the processor recognizes a position of a skeletal joint on a 2D image domain of the 3D high-quality image data.

6. The system of claim 5 , wherein the processor recognizes a 2D position of the skeletal joint, and calculates a 3D position of the skeletal joint using a depth value corresponding to the 2D position and a focal length of a lens.

7. A method of automatically recognizing a user motion, the method comprising the steps of:

learning two-dimensional (2D) user motion to generate a deep learning model;

receiving three-dimensional (3D) measurement data that is acquired by a low-cost depth sensor;

generating 3D low-quality depth data based on the 3D measurement data;

converting the 3D low-quality depth data into 3D high-quality image data; and

performing automatic motion recognition on 3D user motion using the 3D high-quality image data and the deep learning model,

wherein the converting includes synthesizing the 3D low-quality depth data acquired at a current time with 3D high-quality intermediate data generated at an immediately previous time to form 3D high-quality intermediate data that is required to generate the 3D high-quality image data of the current time.

8. The method of claim 7 , wherein the learning includes generating the deep learning model using 2D image data acquired by a camera and 2D user motion data.

9. The method of claim 7 , wherein the generating includes converting the 3D measurement data into the 3D low-quality depth data on a basis of a maximum measurable depth value that is a fixed value for the low-cost depth sensor.

10. The method of claim 7 , wherein the performing includes applying the deep learning model to a 2D image domain of the 3D high-quality image data to perform the automatic motion recognition on the 3D user motion.

11. The method of claim 7 , wherein the performing includes recognizing a 2D position of a skeletal joint, and calculating a 3D position of the skeletal joint using a depth value corresponding to the 2D position and a focal length of a lens.