Image processing apparatus and image processing method
There is provided an image processing apparatus that includes an ML3D model generation section, which are applicable to a home server that generates a display image of a predetermined viewpoint from an omnidirectional image or the like. The ML3D model generation section receives transmission information in which auxiliary information is added to at least one of texture information of a first layer, depth information of the first layer, texture information of a second layer or depth information of the second layer, and executes predetermined image processing using the auxiliary information for at least one of the texture information of the first layer, the depth information of the first layer, the texture information of the second layer or the depth information of the second layer.
1. An image processing apparatus, comprising:
a reception section configured to receive transmission information, wherein
the received transmission information includes auxiliary information and at least one of texture information of a first layer associated with an omnidirectional image, depth information of the first layer associated with the omnidirectional image, texture information of a second layer associated with the omnidirectional image, or depth information of the second layer associated with the omnidirectional image,
the auxiliary information is polygon information that corresponds to a front of a polygon and a back of the polygon,
the polygon corresponds to a viewpoint of the omnidirectional image,
the front of the polygon corresponds to the texture information of the first layer and the depth information of the first layer,
the back of the polygon corresponds to the texture information of the second layer and the depth information of the second layer, and
the polygon information is in at least one of a Cb component or a Cr component of a 4:2:0 YUV format; and
an auxiliary information utilization section configured to execute an image processing operation based on the auxiliary information, wherein the image processing operation is associated with the at least one of the texture information of the first layer, the depth information of the first layer, the texture information of the second layer, or the depth information of the second layer.
2. The image processing apparatus according to claim 1 , wherein
the texture information of the first layer includes a first texture image of the omnidirectional image,
the depth information of the first layer includes a first depth image of the omnidirectional image,
the texture information of the second layer includes a second texture image in an occlusion region at a viewpoint of the first texture image of the first layer, and
the depth information of the second layer includes a second depth image in the occlusion region at the viewpoint of the first texture image of the first layer.
3. The image processing apparatus according claim 1 , wherein
the auxiliary information is in a first hierarchy,
the at least one of the texture information of the first layer, the depth information of the first layer, the texture information of the second layer, or the depth information of the second layer is in a second hierarchy, and
the first hierarchy is same as the second hierarchy.
4. The image processing apparatus according to claim 1 , wherein each of the depth information of the first layer and the depth information of the second layer is in a Y component of the 4:2:0 YUV format.
5. An image processing method, comprising:
receiving transmission information, wherein
the received transmission information includes auxiliary information and at least one of texture information of a first layer associated with an omnidirectional image, depth information of the first layer associated with the omnidirectional image, texture information of a second layer associated with the omnidirectional image, or depth information of the second layer associated with the omnidirectional image,
the auxiliary information is polygon information that corresponds to a front of a polygon and a back of the polygon,
the polygon corresponds to a viewpoint of the omnidirectional image,
the front of the polygon corresponds to the texture information of the first layer and the depth information of the first layer,
the back of the polygon corresponds to the texture information of the second layer and the depth information of the second layer, and
the polygon information is in at least one of a Cb component or a Cr component of a 4:2:0 YUV format; and
executing an image processing operation based on the auxiliary information, wherein the image processing operation is associated with the at least one of the texture information of the first layer, the depth information of the first layer, the texture information of the second layer, or the depth information of the second layer.
6. An image processing apparatus, comprising:
an image generation section configured to generate texture information of a first layer associated with an omnidirectional image, depth information of the first layer associated with the omnidirectional image, texture information of a second layer associated with the omnidirectional image, and depth information of the second layer associated with the omnidirectional image;
an auxiliary information generation section configured to generate auxiliary information to be added to at least one of the texture information of the first layer, the depth information of the first layer, the texture information of the second layer, or the depth information of the second layer, wherein
the auxiliary information is polygon information that corresponds to a front of a polygon and a back of the polygon,
the polygon corresponds to a viewpoint of the omnidirectional image,
the front of the polygon corresponds to the texture information of the first layer and the depth information of the first layer, and
the back of the polygon corresponds to the texture information of the second layer and the depth information of the second layer; and
an element header information setting section configured to set the polygon information in at least one of a Cb component or a Cr component of a 4:2:0 YUV format.
7. The image processing apparatus according to claim 6 , wherein
the texture information of the first layer includes a first texture image of the omnidirectional image,
the depth information of the first layer includes a first depth image of the omnidirectional image,
the texture information of the second layer includes a second texture image in an occlusion region at a viewpoint of the first texture image of the first layer, and
the depth information of the second layer includes a second depth image in the occlusion region at the viewpoint of the first texture image of the first layer.
8. The image processing apparatus according to claim 6 , wherein the element header information setting section is further configured to:
set the auxiliary information in a first hierarchy; and
set the at least one of the texture information of the first layer, the depth information of the first layer, the texture information of the second layer, or the depth information of the second layer in a second hierarchy,
wherein the first hierarchy is same as the second hierarchy.
9. The image processing apparatus according to claim 6 , wherein the element header information setting section is further configured to set the depth information of the first layer and the depth information of the second layer in a Y component of the 4:2:0 YUV format.
10. An image processing method, comprising:
generating texture information of a first layer associated with an omnidirectional image, depth information of the first layer associated with the omnidirectional image, texture information of a second layer associated with the omnidirectional image, and depth information of the second layer associated with the omnidirectional image;
generating auxiliary information to be added to at least one of the texture information of the first layer, the depth information of the first layer, the texture information of the second layer, or the depth information of the second layer, wherein
the auxiliary information is polygon information that corresponds to a front of a polygon and a back of the polygon,
the polygon corresponds to a viewpoint of the omnidirectional image,
the front of the polygon corresponds to the texture information of the first layer and the depth information of the first layer, and
the back of the polygon corresponds to the texture information of the second layer and the depth information of the second layer; and
setting the polygon information in at least one of a Cb component or a Cr component of a 4:2:0 YUV format.