Method and apparatus for processing 360-degree image
A communication technique for merging, with an IoT technology, a 5G communication system for supporting a data transmission rate higher than that of a 4G system is provided. The communication technique can be applied to an intelligent service (for example, smart home, smart building, smart city, smart car or connected car, health care, digital education, retail business, and security and safety-related services, and the like) on the basis of a 5G communication technology and an IoT-related technology. A method for processing a 360-degree image is provided. The method includes determining a three-dimensional (3D) model for mapping a 360-degree image; determining a partition size for the 360-degree image; determining a rotational angle for each of the x, y, and z axes of the 360-degree image; determining an interpolation method to be applied when mapping the 360-degree image to a two-dimensional (2D) image; and converting the 360-degree image into the 2D image.
1. A method for processing 360 -degree multi-view images, the method comprising:
identifying, by a processor of an electronic device, media data for a two-dimensional (2D) image which is generated based on projecting a three-dimensional (3D) image to a planar space, and metadata related to the media data; and
transmitting, by the processor of the electronic device, the identified media data and the identified metadata,
wherein the 3D image is generated based on the 360-degree multi-view images, and
wherein the metadata comprises:
information indicating a 3D model for mapping the 2D image to the 3D model among a plurality of 3D models, and
information indicating a rotating angle according to each of an x-axis, a y-axis, and a z-axis for the 3D image.
2. The method of claim 1 , further comprising:
determining whether a partition size, the rotating angles, and an interpolation method for the 3D image which minimizes distortion at a predetermined bitrate by comparing degrees of distortion for 2D images based on other partition sizes, other rotating angles, and another interpolation method, and a degree of distortion for the 2D images based on the partition size, the rotating angles, and the interpolation method,
wherein the metadata further comprises information indicating the partition size for the 3D image and an interpolation method for the 3D image.
3. The method of claim 2 , further comprising, when the partition size, the rotating angles, and the interpolation method are not determined to minimize the distortion at the predetermined bitrate, re-determining the partition size, the rotating angles, and the interpolation method.
4. The method of claim 3 , wherein the partition size for the 3D image is determined as a value to minimize an entire rate-distortion (RD) cost based on an amount of the metadata and the degree of distortion for the 2D image.
5. The method of claim 1 , further comprising:
determining whether to apply a control grid interpolation method in which control points in partitioned regions of the 3D image are shifted in the direction of the x-axis and/or the y-axis by a predetermined distance to be varied for mapping each corner of the partitioned regions,
wherein the control points are corners of the partitioned regions.
6. The method of claim 1 , further comprising:
mapping non-background contents in the 3D image to relatively less distorted regions in the 2D image; and
mapping background contents in the 3D image to a relatively more distorted regions in the 2D image.
7. A method for processing 360-degree multi-view images, the method comprising:
receiving, by a processor of an electronic device, media data for a two-dimensional (2D) image which is generated based on projecting a three-dimensional (3D) image to a planar space, and metadata related to the media data, wherein the 3D image is generated based on the 360-degree multi-view images; and
rendering, by the processor of the electronic device, the received media data based on the received metadata,
wherein the metadata comprises:
information indicating a 3D model for mapping the 2D image to the 3D model among a plurality of 3D models, and
information indicating a rotating angle according to each of an x-axis, a y-axis, and a z-axis for the 3D image.
8. The method of claim 7 , wherein the rendering of the received media data further comprises:
mapping the 2D image to the 3D model based on the information indicating the 3D model, and
rotating the mapped 2D image on the 3D model based on the information indicating the rotating angles.
9. The method of claim 8 , further comprising:
applying a control grid interpolation method to the 2D image,
wherein the control grid interpolation method is a method in which control points in a partitioned regions are shifted in a direction of the x-axis and/or the y-axis by a predetermined distance to be varied.
10. A device for processing 360-degree multi-view images, the device comprising:
a transceiver; and
a processor coupled to the transceiver, wherein the processor is configured to:
identify media data for a two-dimensional (2D) image which is generated based on projecting a three-dimensional (3D) image to a planar space, and metadata related to the media data, and
transmit the identified media data and the identified metadata, wherein the 3D image is generated based on the 360-degree multi-view images, wherein the metadata comprises:
information indicating a 3D model for mapping the 2D image to the 3D model among a plurality of 3D models, and
information indicating a rotating angle according to each of an x-axis, a y-axis, and a z-axis for the 3D image.
11. The device of claim 10 , wherein the processor is further configured to:
determine whether a partition size, the rotating angles, and an interpolation method minimizes distortion at a predetermined bitrate by comparing degrees of distortion for the 2D image based on other partition sizes, other rotating angles, and another interpolation method, and a degree of distortion for the 2D image based on the partition size, the rotating angles, and the interpolation method,
wherein the metadata further comprises information indicating the partition size for the 3D image and an interpolation method for the 3D image.
12. A device for processing 360 -degree multi-view images, the device comprising:
a transceiver; and
a processor coupled to the transceiver, wherein the processor is configured to:
receive media data for a two-dimensional (2D) image which is generated based on projecting a three-dimensional (3D) image to a planar space, and metadata related to the media data, wherein the 3D image is generated based on the 360-degree multi-view images, and
render the received media data based on the received metadata, wherein the metadata comprises:
information indicating a 3D model for mapping the 2D image to the 3D model among a plurality of 3D models, and
information indicating a rotating angle according to each of an x-axis, a y-axis, and a z-axis for the 3D image.
13. The device of claim 12 , wherein, to render the received media data, the processor is further configured to:
map the 2D image to the 3D model based on the information indicating the 3D model, and
rotate the mapped 2D image on the 3D model based on the information indicating the rotating angles.