Image processing apparatus and method, file generation apparatus and method, and program
There is provided an image processing apparatus and method, a file generation apparatus and method, and a program that enable a suitable occlusion image to be obtained. The image processing apparatus includes an MPD file processing unit configured to select an occlusion image to be acquired, on the basis of information regarding a viewpoint position of the occlusion image included in an MPD file, from among a plurality of the occlusion images indicated by the MPD file. The present technology can be applied to a client device.
1. An image processing apparatus comprising:
an MPD file processing unit configured to select an occlusion image to be acquired, on a basis of information regarding a viewpoint position of the occlusion image included in an MPD file of a main image and information regarding a depth of the occlusion image included in the MPD file, from among a plurality of the occlusion images indicated by the MPD file, wherein
a full spherical image reproduced by texture information of the main image comprises an omnidirectional image viewed from an origin in a predetermined coordinate system,
the occlusion image comprises texture information of an occlusion region of a second full spherical image viewed from a viewpoint other than the origin,
the information regarding the depth of the occlusion image includes information indicating an order of a depth amount of the occlusion region of the occlusion image,
the occlusion region comprises texture information of a subject that cannot be obtained from the texture information of the main image viewed from the origin,
the information regarding the depth of the occlusion image further includes information indicating a distance in a depth direction corresponding to the subject at each position on the occlusion image, and
the MPD file processing unit is implemented via at least one processor.
2. The image processing apparatus according to claim 1 , wherein
the MPD file processing unit is further configured to select the occlusion image of a number according to a transmission band.
3. The image processing apparatus according to claim 1 , further comprising:
an acquisition unit configured to acquire the occlusion image selected by the MPD file processing unit,
wherein the acquisition unit is implemented via at least one processor.
4. The image processing apparatus according to claim 1 , wherein
the MPD file processing unit is further configured to select the occlusion image sequentially from one in which a viewpoint position of the occlusion image is closer to a viewpoint other than the origin.
5. The image processing apparatus according to claim 1 , wherein
the information regarding a viewpoint position of the occlusion image includes coordinate information indicating a viewpoint position of the occlusion image.
6. The image processing apparatus according to claim 1 , wherein
the information regarding a viewpoint position of the occlusion image includes direction information indicating a direction of a viewpoint position of the occlusion image viewed from a reference position.
7. The image processing apparatus according to claim 6 , wherein
the information regarding a viewpoint position of the occlusion image includes the direction information, and distance information indicating a distance from the reference position to a viewpoint position of the occlusion image.
8. The image processing apparatus according to claim 6 , wherein
the information regarding a viewpoint position of the occlusion image includes occlusion direction information, or occlusion direction information and occlusion distance information.
9. The image processing apparatus according to claim 1 , wherein
for each occlusion image of the plurality of the occlusion images, the MPD file includes: selection information indicating which information is included out of the information regarding a viewpoint position of the occlusion image and the information regarding the depth of the occlusion image; and information indicated by the selection information.
10. The image processing apparatus according to claim 1 , wherein
the MPD file processing unit is further configured to select the occlusion image by further using information regarding a priority of the occlusion image.
11. The image processing apparatus according to claim 10 , wherein
the MPD file processing unit is further configured to select the occlusion image for each time by using information regarding a priority of the occlusion image determined for each time.
12. The image processing apparatus according to claim 11 , wherein
for each occlusion image of the plurality of the occlusion images, the MPD file includes:
information indicating a period; and
information regarding a priority of the occlusion image in the period, for each of a plurality of periods.
13. The image processing apparatus according to claim 11 , wherein
information regarding a priority of the occlusion image is included in a file that stores information regarding a quality of a content to be reproduced using the occlusion image.
14. The image processing apparatus according to claim 1 , wherein
the MPD file includes viewpoint position information indicating a range of a viewpoint position of a user in which the occlusion image can be used, and
the MPD file processing unit is further configured to select the occlusion image to be acquired, on a basis of the viewpoint position information, from the plurality of the occlusion images in which a viewpoint position of the user is within the range among the plurality of the occlusion images.
15. The image processing apparatus according to claim 14 , further comprising:
a display information generation unit configured to generate and output display information indicating that a viewpoint position of the user is outside a display range of a content to be reproduced using the occlusion image, in a case where there are none of the plurality of the occlusion images in which a viewpoint position of the user is within the range,
wherein the display information generation unit is implemented via at least one processor.
16. The image processing apparatus according to claim 1 , wherein the order of the depth amount is determined based on a sum of depth values at individual positions in a predetermined frame of the texture information the occlusion image.
17. An image processing method comprising:
selecting an occlusion image to be acquired, on a basis of information regarding a viewpoint position of the occlusion image included in an MPD file of a main image and information regarding a depth of the occlusion image included in the MPD file, from among a plurality of the occlusion images indicated by the MPD file, wherein
a full spherical image reproduced by texture information of the main image comprises an omnidirectional image viewed from an origin in a predetermined coordinate system,
the occlusion image comprises texture information of an occlusion region of a second full spherical image viewed from a viewpoint other than the origin,
the information regarding the depth of the occlusion image includes information indicating an order of a depth amount of the occlusion region of the occlusion image,
the occlusion region comprises texture information of a subject that cannot be obtained from the texture information of the main image viewed from the origin, and
the information regarding the depth of the occlusion image further includes information indicating a distance in a depth direction corresponding to the subject at each position on the occlusion image.
18. A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the computer to execute an image processing method, the method comprising:
selecting an occlusion image to be acquired, on a basis of information regarding a viewpoint position of the occlusion image included in an MPD file of a main image and information regarding a depth of the occlusion image included in the MPD file, from among a plurality of the occlusion images indicated by the MPD file, wherein
a full spherical image reproduced by texture information of the main image comprises an omnidirectional image viewed from an origin in a predetermined coordinate system,
the occlusion image comprises texture information of an occlusion region of a second full spherical image viewed from a viewpoint other than the origin,
the information regarding the depth of the occlusion image includes information indicating an order of a depth amount of the occlusion region of the occlusion image,
the occlusion region comprises texture information of a subject that cannot be obtained from the texture information of the main image viewed from the origin, and
the information regarding the depth of the occlusion image further includes information indicating a distance in a depth direction corresponding to the subject at each position on the occlusion image.
19. A file generation apparatus comprising:
an MPD file generation unit configured to generate an MPD file of a main image including information regarding a viewpoint position of an occlusion image and information regarding a depth of the occlusion image for a plurality of the occlusion images; and
a communication unit configured to send the MPD file, wherein
a full spherical image reproduced by texture information of the main image comprises an omnidirectional image viewed from an origin in a predetermined coordinate system,
the occlusion image comprises texture information of an occlusion region of a second full spherical image viewed from a viewpoint other than the origin,
the information regarding the depth of the occlusion image includes information indicating an order of a depth amount of the occlusion region of the occlusion image,
the occlusion region comprises texture information of a subject that cannot be obtained from the texture information of the main image viewed from the origin,
the information regarding the depth of the occlusion image further includes information indicating a distance in a depth direction corresponding to the subject at each position on the occlusion image, and
the MPD file generation unit and the communication unit are each implemented via at least one processor.
20. A file generation method comprising:
generating an MPD file of a main image including information regarding a viewpoint position of an occlusion image and information regarding a depth of the occlusion image for a plurality of the occlusion images; and
sending the MPD file, wherein
a full spherical image reproduced by texture information of the main image comprises an omnidirectional image viewed from an origin in a predetermined coordinate system,
the occlusion image comprises texture information of an occlusion region of a second full spherical image viewed from a viewpoint other than the origin,
the information regarding the depth of the occlusion image includes information indicating an order of a depth amount of the occlusion region of the occlusion image,
the occlusion region comprises texture information of a subject that cannot be obtained from the texture information of the main image viewed from the origin, and
the information regarding the depth of the occlusion image further includes information indicating a distance in a depth direction corresponding to the subject at each position on the occlusion image.
21. A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the computer to execute a file generation method, the method comprising:
generating an MPD file of a main image including information regarding a viewpoint position of an occlusion image and information regarding a depth of the occlusion image for a plurality of the occlusion images; and
sending the MPD file, wherein
a full spherical image reproduced by texture information of the main image comprises an omnidirectional image viewed from an origin in a predetermined coordinate system,
the occlusion image comprises texture information of an occlusion region of a second full spherical image viewed from a viewpoint other than the origin,
the information regarding the depth of the occlusion image includes information indicating an order of a depth amount of the occlusion region of the occlusion image,
the occlusion region comprises texture information of a subject that cannot be obtained from the texture information of the main image viewed from the origin, and
the information regarding the depth of the occlusion image further includes information indicating a distance in a depth direction corresponding to the subject at each position on the occlusion image.