Generation method for a steerable realistic image contents and motion simulation system thereof

1 . A device for generating steerable realistic image content, comprising:

at least one processor; and

a memory configured to store at least one program which is executed by the at least one processor,

wherein the processor is configured to:

acquire a motion path in a realistic image captured by a camera while the camera is moving along a predetermined path, wherein the motion path is extracted from the realistic image using Visual Simultaneous Localization And Mapping (SLAM), and is generated reflecting geographic information system (GIS) information of a location at which the realistic image is captured,

generate a 3-dimensional (3D) motion model based on the motion path, and

synchronize the 3D motion model with the realistic image to generate synchronization information,

wherein the 3D motion model includes a 3D moving object which moves along a 3D motion path generated along 3D coordinates of the motion path, wherein the 3D motion path is formed by a movement of a cylindrical model along the 3D coordinates of the motion path in a sequential order and a movement of the 3D moving object is modeled reflecting physical features applied to the 3D moving object as the 3D moving object moves along the motion path, wherein a change in direction of the 3D moving object is limited to a range of the realistic image,

wherein the realistic image includes a plurality of images generated frame by frame, and a reference point corresponding to the moving object to be modeled in 3D is defined for each of the plurality of images,

wherein the motion path is generated by arranging the plurality of images on a 3D space in a playing order, and tracking a change in 3D coordinates of the reference point found on the plurality of arranged images,

wherein the movement of the 3D moving object and the plurality of images for each frame is matched with respect to a plurality of key frames defined at a predetermined interval between a start frame and an end frame of the plurality of images,

wherein the processor is configured to determine a location at which the 3D motion model displayed with the realistic image is displayed on a display, wherein the location corresponds to the reference point of the realistic image,

wherein the processor is configured to synchronize at least a movement speed of the 3D moving object with a playing speed of the realistic image, and

wherein the movement speed of the 3D moving object changes in response to a change in the playing speed of the realistic image, and the playing speed of the realistic image changes in response to a change in the movement speed of the 3D moving object caused by a first user's manipulation.

2 . The device for generating steerable realistic image content according to claim 1 , wherein the camera is a 360-degree camera, and the realistic image is a 360-degree image.

3 . The device for generating steerable realistic image content according to claim 1 , wherein the processor is further configured to convert a first-person view (FPV) image transmitted from the camera to the realistic image, and the realistic image is a 180-degree image.

4 . The device for generating steerable realistic image content according to claim 3 , wherein the realistic image is defined as a focused region and a surrounding region outside of the focused region, and

wherein the processor is further configured to carry out blurring to make the surrounding region blurry than the focused region.

5 . A motion simulation system of steerable realistic image content, comprising:

a device for generating realistic image content to generate the steerable realistic image content; and

a device for providing a realistic image to provide the steerable realistic image content to a first user,

wherein the device for generating realistic image content includes at least one first processor and a first memory to store at least one program which is executed by the at least one first processor,

wherein the first processor is configured to: acquire a motion path in a realistic image captured by a camera while the camera is moving along a predetermined path, wherein the motion path is extracted from the realistic image using Visual Simultaneous Localization And Mapping (SLAM), and is generated reflecting geographic information system (GIS) information of a location at which the realistic image is captured, generate a 3-dimensional (3D) motion model based on the motion path, and synchronize the 3D motion model with the realistic image to generate synchronization information,

wherein the 3D motion model includes a 3D moving object which moves a 3D motion path generated along 3D coordinates of the motion path,

wherein the 3D motion path is formed by a movement of a cylindrical model along the 3D coordinates of the motion path in a sequential order,

wherein a movement of the 3D moving object is modeled reflecting physical features applied to the 3D moving object as the 3D moving object moves along the motion path,

wherein a change in direction of the 3D moving object is limited to a range of the realistic image,

wherein the steerable realistic image content includes the realistic image, the 3D motion model and the synchronization information,

wherein the device for providing a realistic image includes:

at least one second processor and a second memory to store at least one program which is executed by the at least one second processor, wherein the second processor is configured to input the first user's steering command,

at least one third processor and a third memory to store at least one program which is executed by the at least one third processor, wherein the third processor is configured to output the steerable realistic image content to the first user,

at least one fourth processor and a fourth memory to store at least one program which is executed by the at least one fourth processor, wherein the fourth processor is configured to transmit a motion change in the steerable realistic image content to the first user, and

a fifth processor configured to control the second processor, the third processor and the fourth processor,

wherein the movement of the 3D moving object and the plurality of images for each frame is matched with respect to a plurality of key frames defined at a predetermined interval between a start frame and an end frame of the plurality of images,

wherein the first processor is configured to determine a location at which the 3D motion model displayed with the realistic image is displayed on a display, wherein the location corresponds to the reference point of the realistic image,

wherein the fifth processor is configured to provide the realistic image to the first user through the third processor, and provide the steerable realistic image content to the first user by simulating the 3D motion model corresponding to the realistic image with reference to the synchronization information,

wherein the fifth processor is configured to perform processing to reflect the first user's input provided by the second processor on the steerable realistic image content in real time,

wherein when the first user changes a speed of the 3D moving object through the second processor, the fifth processor is configured to simulate the 3D moving object to match the changed speed, change the playing speed of the realistic image in sync and provide to the first user, and

wherein the fifth processor is configured to change the movement speed of the 3D moving object in response to a change in the playing speed of the realistic image, and to change the playing speed of the realistic image in response to a change in the movement speed of the 3D moving object caused by a first user's manipulation.

6 . The motion simulation system of steerable realistic image content according to claim 5 , wherein the first processor is further configured to convert a first-person view (FPV) image transmitted from the camera to the realistic image, and the realistic image is a 180-degree image.

7 . The motion simulation system of steerable realistic image content according to claim 6 , wherein the realistic image is defined as a focused region and a surrounding region outside of the focused region, and

wherein the fifth processor is further configured to receive information associated with a change in head direction of the first user from the third processor, define the focused region and the surrounding region in real time based on the received information associated with the change in head direction and blur the defined surrounding region.

8 . A method for generating steerable realistic image content, performed by a device for generating steerable realistic image content, including at least one processor and a memory for storing a program which is executed by the processor, the method comprising:

acquiring a motion path in a realistic image captured by a camera while the camera is moving along a predetermined path, wherein the motion path is extracted from the realistic image using Visual Simultaneous Localization And Mapping (SLAM), and is generated reflecting geographic information system (GIS) information of a location at which the realistic image is captured;

generating a 3-dimensional (3D) motion model based on the motion path, wherein the 3D motion model includes a 3D moving object which moves along a 3D motion path generated along 3D coordinates of the motion path, wherein the 3D motion path is formed by a movement of a cylindrical model along the 3D coordinates of the motion path in a sequential order; and

synchronizing the 3D motion model with the realistic image,

wherein a movement of the 3D moving object is modeled reflecting physical features applied to the 3D moving object as the 3D moving object moves along the motion path, wherein a change in direction of the 3D moving object is limited to a range of the realistic image,

wherein the realistic image includes a plurality of images generated frame by frame, and a reference point corresponding to the moving object to be modeled in 3D is defined for each of the plurality of images,

wherein the motion path is generated by arranging the plurality of images on a 3D space in a playing order, and tracking a change in 3D coordinates of the reference point found on the plurality of arranged images,

wherein synchronizing the 3D motion model with the realistic image comprises synchronizing at least a movement speed of the 3D moving object with a playing speed of the realistic image,

wherein the movement of the 3D moving object and the plurality of images for each frame is matched with respect to a plurality of key frames defined at a predetermined interval between a start frame and an end frame of the plurality of images,

wherein the synchronizing includes determining a location at which the 3D motion model displayed with the realistic image is displayed on a display, wherein the location corresponds to the reference point of the realistic image, and

wherein the movement speed of the 3D moving object changes in response to a change in the playing speed of the realistic image, and the playing speed of the realistic image changes in response to a change in the movement speed of the 3D moving object caused by a first user's manipulation.

9 . The method for generating steerable realistic image content according to claim 8 , wherein the camera is a 360-degree camera, and the realistic image is a 360-degree image.

10 . The method for generating steerable realistic image content according to claim 8 , further comprising:

generating the realistic image by converting a first-person view (FPV) image transmitted from the camera to the realistic image,

wherein the realistic image is a 180-degree image.

11 . The method for generating steerable realistic image content according to claim 10 , wherein generating the realistic image by converting the FPV image transmitted from the camera to the realistic image further comprises:

defining the realistic image as a focused region and a surrounding region outside of the focused region; and

carrying out blurring to make the surrounding region more blurry than the focused region.

12 . A non-transitory computer-readable medium for storing a computer program which is executed to perform the method for generating steerable realistic image content according to claim 8 .