Image processing apparatus and method with setting of prohibited region and generation of computer graphics data based on prohibited region and first or second position/orientation
This invention has as its object to provide a high-quality composite image of a real image and virtual image to a viewer or the like. To achieve this object, the position/posture of photographing means for photographing a performer or the like is measured, the position/posture of an HMD, which the performer or the like wears, and which allows the viewer to see through the external world, is measured, a CG at the viewpoint of the photographing means and image data photographed by the photographing means are aligned and composited on the basis of the measurement result of the position/posture of the photographing means, CG at the viewpoint of the HMD and image data photographed by a camera built in the HMD are aligned and composited on the basis of the measurement result of the position/posture of the HMD, and image data is displayed on the HMD.
1. An image processing apparatus comprising:
a first measurement unit adapted to measure a first position/orientation of a photographing unit adapted to photograph an object;
a second measurement unit adapted to measure a second position/orientation of the object;
a prohibited region setting unit adapted to set a prohibited region in association with a layout position of the CG (Computer Graphics) data based on the second position/orientation;
a CG data generation unit adapted to generate CG data based on the first or the second position/orientation, and the prohibited region;
a first composition unit adapted to composite the CG data corresponding to the first position/orientation and image data photographed by the photographing unit;
a second composition unit adapted to composite the CG data corresponding to the second position/orientation and image data of a real image corresponding to the second position/orientation;
a selection unit adapted to select image data from image data composited by said first and second composition units; and
a multi channel control unit adapted to generate output data, corresponding to each channel of multiple channels, from the selected image data,
wherein the multiple channels include at least the Internet, a satellite broadcast, and a ground wave broadcast.
2. An image processing apparatus comprising:
a first measurement unit adapted to measure a first position/orientation of a photographing unit adapted to photograph an object;
a second measurement unit adapted to measure a second position/orientation of the object;
a prohibited region setting unit adapted to set a prohibited region in association with a layout position of the CG (Computer Graphics) data based on the second position/orientation;
a CG data generation unit adapted to generate CG data based on the first or the second position/orientation, and the prohibited region;
a selection unit adapted to select CG data from CG data generated by said CG data generation unit; and
a multi channel control unit adapted to generate output data, corresponding to each channel of multiple channels, from the selected CG data,
wherein the multiple channels include at least the Internet, a satellite broadcast, and a ground wave broadcast.
3. The apparatus according to claim 1 , further comprising a Head Mount Display for displaying a second composited image formed by said second composition unit.
4. The apparatus according to claim 1 , further comprising a scenario management unit adapted to manage a scenario
and wherein the CG data generation unit generates the CG data on the basis of the managed scenario.
5. An image processing apparatus comprising:
a first measurement unit adapted to measure a first position/orientation of a photographing unit adapted to photograph an object;
a second measurement unit adapted to measure a second position/orientation of the object;
a prohibited region setting unit adapted to set a prohibited region in association with a layout position of the CG (Computer Graphics) data based on the second position/orientation;
a scenario management unit adapted to manage a scenario;
a CG data generation unit adapted to generate CG data based on the first or the second position/orientation, and the prohibited region;
a first composition unit adapted to composite the CG data corresponding to the first position/orientation and the image data photographed by the photographing unit;
a second composition unit adapted to composite the CG data corresponding to the second position/orientation and the image data of a real image corresponding to the second position/orientation;
a server unit adapted to receive a request from a viewer; and
an operator unit adapted to reflect the request from the viewer received by said server unit in the scenario managed by said scenario management unit.
6. The apparatus according to claim 5 , wherein the request from the viewer is assigned to a plurality of CG data or a predetermined building component of sole CG data for each viewer.
7. The apparatus according to claim 5 , wherein said server unit comprises a limitation unit adapted to limit a viewer from which a request is received.
8. The apparatus according to claim 1 , further comprising an introduction unit adapted to externally introduce the CG data.
9. The apparatus according to claim 1 , wherein CG data associated with an advertisement is included as one of the CG data.
10. The apparatus according to claim 9 , further comprising a change unit adapted to change the CG data associated with the advertisement.
11. The apparatus according to claim 1 , wherein said selection unit selects pre-set image data from image data composited by said first and second composition units.
12. The apparatus according to claim 1 , wherein said selection unit selects image data to be converted into the multi-channel data on the basis of information described in a scenario.
13. The apparatus according to claim 1 , wherein said selection unit selects image data to be converted into the multi-channel data in accordance with a request from a viewer.
14. A computer-implemented image processing method comprising the steps of:
measuring a first position/orientation of a photographing unit adapted to photograph an object;
measuring a second position/orientation of the object;
setting a prohibited region in association with a layout position of the CG (Computer Graphics) data based on the second position/orientation;
generating CG data based on the first or the second position/orientation, and the prohibited region;
compositing the CG data corresponding to the first position/orientation and image data photographed by the photographing unit;
compositing the CG data corresponding to the second position/orientation and image data of a real image corresponding to the second position/orientation;
selecting image data from image data composited by said first position/orientation compositing step and said second position/orientation compositing step; and
generating output data, corresponding to each channel of multiple channels, from the selected image data,
wherein the multiple channels include at least the Internet, a satellite broadcast, and a ground wave broadcast.
15. A control program tangibly embodied on a computer-readable medium for making a computer perform a method comprising the steps of:
measuring a first position/orientation of a photographing unit adapted to photograph an object;
measuring a second position/orientation of the object;
setting a prohibited region in association with a layout position of the CG (Computer Graphics) data based on the second position/orientation;
generating CG data based on the first or the second position/orientation, and the prohibited region;
compositing the CG data corresponding to the first position/orientation and image data photographed by the photographing unit;
compositing the CG data corresponding to the second position/orientation and image data of a real image corresponding to the second position/orientation;
selecting image data from image data composited by said first position/orientation compositing step and said second position/orientation compositing step; and
generating output data, corresponding to each channel of multiple channels, from the selected image data,
wherein the multiple channels include at least the Internet, a satellite broadcast, and a ground wave broadcast.
16. A computer-implemented image processing method comprising the steps of:
measuring a first position/orientation of a photographing unit adapted to photograph an object;
measuring a second position/orientation of the object;
setting a prohibited region in association with a layout position of the CG (Computer Graphics) data based on the second position/orientation;
managing a scenario;
generating CG data based on the first or the second position/orientation, and the prohibited region;
compositing the CG data corresponding to the first position/orientation and the image data photographed by the photographing unit;
compositing the CG data corresponding to the second position/orientation and the image data of a real image corresponding to the second position/orientation;
receiving a request from a viewer; and
operating to reflect the request from the viewer received in said receiving step in the scenario managed in said managing step.
17. A control program tangibly embodied on a computer-readable medium for making a computer perform a method comprising the steps of:
measuring a first position/orientation of a photographing unit adapted to photograph an object;
measuring a second position/orientation of the object;
setting a prohibited region in association with a layout position of the CG (Computer Graphics) data based on the second position/orientation;
managing a scenario;
generating CG data based on the first or the second position/orientation, and the prohibited region;
compositing the CG data corresponding to the first position/orientation and the image data photographed by the photographing unit;
compositing the CG data corresponding to the second position/orientation and the image data of a real image corresponding to the second position/orientation;
receiving a request from a viewer; and
operating to reflect the request from the viewer received in said receiving step in the scenario managed in said managing step.