System and method for video call using augmented reality
An augmented reality-based video calling system includes a first terminal configured to capture a video and a second terminal configured to add virtual content to the video captured by the first terminal and provide information regarding the added virtual content to the first terminal. The first terminal calculates 3D position coordinates of the virtual content in a 3D space corresponding to the video on the basis of the provided information regarding the virtual content, combines the virtual content with the video using the calculated 3D position coordinates, and display the virtual content combined with the video.
1. An augmented reality-based video calling system comprising:
a first terminal configured to capture an image and transmit the image; and
a second terminal configured to receive the image transmitted by the first terminal, add virtual content to the image and provide information regarding the added virtual content to the first terminal,
wherein the first terminal receives the information provided by the second terminal, calculates three-dimensional (3D) position coordinates of the virtual content in a 3D space corresponding to the image on the basis of the information, combines the virtual content with the image using the calculated 3D position coordinates, and displays the virtual content combined with the image
wherein the first terminal generates a 3D spatial map from the image, selects N feature points adjacent to two-dimensional (2D) position coordinates of the virtual content in the image from among feature points in the 3D spatial map, N is a natural number greater than or equal to 1, and calculates a depth of a point corresponding to the 2D position coordinates from depths of the selected N feature points.
2. The augmented reality-based video calling system of claim 1 , wherein the information regarding the virtual content includes information regarding a type of the virtual content and the 2D position coordinates.
3. The augmented reality-based video calling system of claim 2 , wherein the first terminal combines the 2D position coordinates with the calculated depth to generate the 3D position coordinates.
4. The augmented reality-based video calling system of claim 3 , wherein the first terminal recalculates the 3D position coordinates at predetermined intervals.
5. An augmented reality-based video calling method, which is performed by a computing apparatus including one or more processors and a memory configured to store one or more programs executed by the one or more processors, the augmented reality-based video calling method comprising:
capturing an image and transmitting the captured image to a partner terminal for video calling;
receiving information regarding virtual content added to the image from the partner terminal;
calculating three-dimensional (3D) position coordinates of the virtual content in a 3D space corresponding to the image on the basis of the received information regarding the virtual content; and
combining the virtual content with the image using the calculated 3D position coordinates and displaying the virtual content combined with the image
wherein the calculating of 3D position coordinates comprises:
generating a 3D spatial map from the image; and
selecting N feature points adjacent to two-dimensional (2D) position coordinates of the virtual content in the image from among feature points in the 3D spatial map, where N is a natural number greater than or equal to 1, and calculating a depth of a point corresponding to the 2D position coordinates from depths of the selected N feature points.
6. The augmented reality-based video calling method of claim 5 , wherein the information regarding the virtual content includes information regarding a type of the virtual content and the 2D position coordinates.
7. The augmented reality-based video calling method of claim 6 , wherein the calculating of 3D position coordinates comprises:
combining the 2D position coordinates with the calculated depth to generate the 3D position coordinates.
8. A computer program stored in a non-transitory computer-readable storage medium, the computer program comprising:
one or more instructions, wherein, when the instructions are executed by a computing apparatus having one or more processors, the instructions enable the computing apparatus to perform operations of:
capturing an image and transmitting the captured image to a partner terminal for video calling;
receiving information regarding virtual content added to the image from the partner terminal;
calculating three-dimensional (3D) position coordinates of the virtual content in a 3D space corresponding to the image on the basis of the received information regarding the virtual content; and
combining the virtual content with the image using the calculated 3D position coordinates and displaying the virtual content combined with the image
wherein the calculating of 3D position coordinates comprises:
generating a 3D spatial map from the image; and
selecting N feature points adjacent to two-dimensional (2D) position coordinates of the virtual content in the image from among feature points in the 3D spatial map, where N is a natural number greater than or equal to 1, and calculating a depth of a point corresponding to the 2D position coordinates from depths of the selected N feature points.