Generating an augmented reality image using a blending factor
A method for generating an augmented reality image from first and second images, wherein at least a portion of at least one of the first and the second image is captured from a real scene, identifies a confidence region in which a confident determination as to which of the first and second image to render in that region of the augmented reality image can be made, and identifies an uncertainty region in which it is uncertain as to which of the first and second image to render in that region of the augmented reality image. At least one blending factor value in the uncertainty region is determined based upon a similarity between a first colour value in the uncertainty region and a second colour value in the confidence region, and an augmented reality image is generated by combining, in the uncertainty region, the first and second images using the at least one blending factor value.
1. A method for generating a composite image from first and second images, wherein at least a portion of at least one of the first image and second image is captured from a real scene: the first and second images each comprising a first region and a second region; the method comprising:
I) determining at least one blending factor value, for use in generating a first region of the composite image, based upon a similarity in one of the first and second images between a first value in the first region and a second value in the second region; wherein:
a) the first region of the composite image corresponds to the respective first regions of each of the first and second image;
b) a determination as to which of the first and second images to render can be made for a second region of the composite image; the second region of the composite image corresponds to the respective second regions of each of the first and second image; and
II) generating the composite image by combining, in the first region of the composite image, the first and second images using the at least one blending factor value.
2. The method of claim 1 , wherein the first region of the composite image is an uncertainty region in which it is uncertain as to which of the first and second image to render in that region of the composite image.
3. The method of claim 1 , wherein the second region of the composite image is a confidence region in which a confident determination as to which of the first and second image to render in that region of the composite image can be made.
4. The method of claim 1 , wherein the first and second values are colour values.
5. The method of claim 1 , further comprising:
identifying the first region of the composite image; and
identifying the second region of the composite image.
6. The method of claim 1 , wherein the first image and the second image each have associated therewith a plurality of colour values and a corresponding plurality of depth values, wherein the method further comprises making said determination as to which of the first image and the second image to render based upon a depth value of the first image and the corresponding depth value of the second image in the second region.
7. The method of claim 5 , wherein the first image and the second image each have associated therewith a plurality of colour values and a corresponding plurality of depth values, and wherein the first region is identified based upon at least one depth value associated with at least one of the first and the second image, the at least one depth value being derived from a depth value captured from the real scene.
8. The method of claim 1 , further comprising generating at least one initial blending factor value in a second region based upon said determination and wherein generating the composite image further comprises combining a corresponding colour value of the first image and a corresponding colour value of the second image in the second region using the at least one initial blending factor value.
9. The method of claim 3 , further comprising identifying the confidence region, wherein identifying the confidence region comprises categorising portions of the confidence region as first confidence regions or second confidence regions, wherein:
first confidence regions are confidence regions in which a colour value of the first image is to be rendered in the corresponding region of the composite image; and
second confidence regions are confidence regions in which a colour value of the second image is to be rendered in the corresponding region of the composite image.
10. The method of claim 1 , wherein determining the at least one blending factor value is further based upon the distance between the position of the first value and the position of the second value.
11. The method of claim 1 , wherein the first region, of the at least one of the first image and second image that is captured from a real scene, comprises a plurality of sample points and determining the at least one blending factor value further comprises processing, for each of a plurality of sample points in the said first region, that sample point based upon values at a plurality of sample points located in a second region, of the said at least one of the first image and second image, within a predetermined distance of that sample point.
12. The method of claim 1 , wherein determining at least one blending factor value for the first region of the composite image is based upon a similarity is between a colour value in the said first region and at least one corresponding colour value of each of the first image and the second image.
13. The method of claim 1 , further comprising performing an erosion operation on the second region of the composite image, wherein the erosion operation is configured to re-categorise at least one portion of the second region of the composite image as forming a part of a first region of the composite image.
14. The method of claim 1 , wherein the first image is a captured image of a real scene and the second image is an image of a virtual object.
15. An image processing system for generating a composite image from first and second images, wherein at least a portion of at least one of the first image and second image is captured from a real scene: the first and second images each comprising a first region and a second region; the image processing system comprising:
I) a blend module arranged to determine at least one blending factor value, for use in generating a first region of the composite image, based upon a similarity in one of the first and second images between a first value in the first region and a second value in the second region; wherein:
a) the first region of the composite image corresponds to the respective first regions of each of the first and second image;
b) a determination as to which of the first and second images to render can be made for a second region of the composite image; the second region of the composite image corresponds to the respective second regions of each of the first and second image; and
II) an image generation module arranged to generate the composite image by combining, in the first region of the composite image, the first and second images using the at least one blending factor value.
16. The image processing system of claim 15 , wherein the second region of the composite image is a confidence region in which a confident determination as to which of the first and second image to render in that region of the composite image can be made, and wherein the image processing system further comprises a confidence identification module arranged to identify the confidence region.
17. The image processing system of claim 15 , wherein the first region of the composite image is an uncertainty region in which it is uncertain as to which of the first and second image to render in that region of the composite image, and wherein the image processing system further comprises an uncertainty identification module arranged to identify the uncertainty region.
18. The image processing system of claim 15 , wherein the first image and the second image each have associated therewith a plurality of colour values and a corresponding plurality of depth values, and wherein the first and second values are colour values.
19. The image processing system of claim 15 , wherein the image processing system is embodied in hardware on an integrated circuit.
20. A non-transitory computer readable storage medium having stored thereon computer readable instructions that, when executed at a computer system, cause the computer system to generate a composite image from first and second images, wherein at least a portion of at least one of the first image and second image is captured from a real scene: the first and second images each comprise a first region and a second region; the computer system caused to generate the composite image by:
I) determining at least one blending factor value, for use in generating a first region of the composite image, based upon a similarity in one of the first and second images between a first value in the first region and a second value in the second region; wherein:
a) the first region of the composite image corresponds to the respective first regions of each of the first and second image;
b) a determination as to which of the first and second images to render can be made for a second region of the composite image; the second region of the composite image corresponds to the respective second regions of each of the first and second image; and
II) generating the composite image by combining, in the first region of the composite image, the first and second images using the at least one blending factor value.