Image generation system, image generation method, and information storage medium
View Patent ↗An image generation system includes a captured image acquisition section that acquires a captured image captured by an imaging section, a depth information acquisition section that acquires depth information about a photographic object observed within the captured image, an object processing section that performs a process that determines a positional relationship between the photographic object and a virtual object in a depth direction based on the acquired depth information, and synthesizes the virtual object with the captured image, and an image generation section that generates an image in which the virtual object is synthesized with the captured image.
1. An image generation system comprising:
a captured image acquisition section that acquires a captured image captured by an imaging section;
a captured image storage section that stores image data about a plurality of live-view images captured by the imaging section;
a depth information acquisition section that acquires depth information about a photographic object observed within the captured image, and depth information about a placement position determination photographic object for determining a placement position of a virtual object based on the plurality of live-view images;
an object processing section that performs a process that:
determines a positional relationship between the photographic object and the virtual object in a depth direction based on the acquired depth information about the photographic object and the depth information about the placement position determination photographic object acquired from the plurality of live-view images, and
synthesizes the virtual object with the captured image based on the determined positional relationship by generating an image in which the virtual object is synthesized with the captured image such that the photographic object is displayed on a front side of the virtual object, when it has been determined that the photographic object is positioned on the front side of the virtual object in the depth direction; and
an image generation section that generates an image in which the virtual object is synthesized with the captured image, wherein
the synthesizing of the virtual object with the captured image comprises performing hidden surface removal of the virtual object and translucent blending, and
the hidden surface removal removes hidden surfaces of the virtual object from the display when a physical object is in front of the virtual object.
2. The image generation system as defined in claim 1 ,
the depth information acquisition section acquiring the depth information about the placement position determination photographic object based on a live-view image among the plurality of live-view images, in which it has been determined that a placement position determination area of the placement position determination photographic object is not hidden behind the photographic object.
3. The image generation system as defined in claim 1 , wherein the object processing section further performs a hit determination process on the photographic object and the virtual object based on the depth information.
4. The image generation system as defined in claim 3 , wherein the object processing section:
sets a hit volume for the virtual object based on the depth information about the placement position determination photographic object for determining the placement position of the virtual object, and
performs the hit determination process on the photographic object and the virtual object using the set hit volume.
5. The image generation system as defined in claim 3 ,
the object processing section further performing at least one of a process that changes a display state of the virtual object and a process that outputs sound corresponding to the virtual object when it has been determined that the photographic object has hit the virtual object by the hit determination process.
6. The image generation system as defined in claim 1 ,
the object processing section further performing a movement process of the virtual object based on the depth information.
7. The image generation system as defined in claim 6 , wherein the object processing section moves the virtual object between a position of the placement position determination photographic object for determining the placement position of the virtual object and a position of the photographic object based on the depth information about the placement position determination photographic object and the depth information about the photographic object.
8. The image generation system as defined in claim 1 ,
the captured image acquisition section acquiring a first captured image captured by a first camera included in the imaging section as a left-eye image, and acquiring a second captured image captured by a second camera included in the imaging section as a right-eye image, and
the depth information acquisition section acquiring the depth information based on parallax information obtained from the left-eye image and the right-eye image.
9. The image generation system as defined in claim 8 , wherein the image generation section generates the image in which the virtual object is synthesized with a stereoscopic image generated using the left-eye image and the right-eye image.
10. The image generation system as defined in claim 1 ,
the captured image acquisition section acquiring a depth image from a depth camera included in the imaging section, and
the depth information acquisition section acquiring the depth information based on the depth image.
11. An image generation method comprising:
acquiring a captured image captured by an imaging section;
storing image data about a plurality of live-view images captured by the imaging section;
acquiring depth information about a photographic object observed within the captured image, and depth information about a placement position determination photographic object for determining a placement position of a virtual object based on the plurality of live-view images;
performing a process that determines a positional relationship between the photographic object and the virtual object in a depth direction based on the acquired depth information about the photographic object and the depth information about the placement position determination photographic object acquired from the plurality of live-view images; and
synthesizing the virtual object with the captured image based on the determined positional relationship by generating an image in which the virtual object is synthesized with the captured image such that the photographic object is displayed on a front side of the virtual object, when it has been determined that the photographic object is positioned on the front side of the virtual object in the depth direction;
wherein
the synthesizing of the virtual object with the captured image comprises performing hidden surface removal of the virtual object and translucent blending, and
the hidden surface removal removes hidden surfaces of the virtual object from the display when a physical object is in front of the virtual object.
12. The image generation method as defined in claim 11 , further comprising: acquiring the depth information about the placement position determination photographic object based on a live-view image among the plurality of live-view images, the live-view image being a live-view image in which it has been determined that a placement position determination area of the placement position determination photographic object is not hidden behind the photographic object.
13. A non-transitory computer-readable information storage medium storing a program that causes a computer to execute the image generation method as defined in claim 11 .
14. The image generation system of claim 1 , wherein the positional relationship in the depth direction is a relationship that indicates whether the photographic object is positioned on the front side, the rear side, or overlaps the virtual object when viewed from the imaging section.
15. The image generation system of claim 1 , further comprising a translucent blending section that blends an area where the photographic object overlaps the virtual object, when an area of the photographic object overlaps an area of the virtual object.
16. The image generation system of claim 1 , wherein the virtual object is one of: a human, an animal, a robot, a car, a ship, or an airplane.
17. The image generation method of claim 11 , wherein the object processing section performs a movement process on the virtual object based on the depth information.
18. The image generation method of claim 11 , wherein
the object processing section comprises a hit determination process that determines whether a physical object has hit the virtual object, and
when a physical object has hit the virtual object, the object processing section performs at least one of changing a display state of the virtual object and outputting a sound corresponding to the virtual object.