Direct light compensation technique for augmented reality system

1. A method of operating an augmented reality (AR) system, the method comprising:

warping first image data into warped image data based at least in part a perspective of a user and a position of the capturing device, wherein the first image data is captured by a capturing device from direct light in a three-dimensional scene;

determining an overlap region between a real object in the three-dimensional scene and a virtual object in virtual image data; and

presenting a three-dimensional augmented scene to the user at least by:

determining a characteristic of the real object in the overlap region;

identifying a pixel to be presented in the three-dimensional augmented scene to the user;

identifying real object data for displaying the characteristic of the real object at the pixel from the warped image data;

identifying virtual object data for displaying a corresponding portion of the virtual object at the pixel from the virtual image data;

determining pixel interference data for displaying the three-dimensional augmented scene at the pixel based at least in part upon the first data and the virtual object data; and

displaying, along with the direct light, the three-dimensional augmented scene to the user at least by adding the real object data, the pixel interference data, and the virtual object data for the pixel so that that the characteristic of the real object appears to be completely obscured by the virtual object from the perspective of the user;

wherein displaying the three-dimensional augmented scene comprises

generating the pixel interference data is based at least in part on the characteristic of the real object in the overlap region; and

displaying the pixel interference data as an interference image over the overlap region to the user such that the interference image combines with the direct light from the real object to create a background for the virtual object in the overlap region.

2. The method of claim 1 , wherein the background in the overlap region comprises a decreased dynamic color range relative to the real object in the overlap region.

3. The method of claim 2 , wherein the background has a generally uniform color in the overlap region.

4. The method of claim 3 , wherein the generally uniform color is grey.

5. The method of claim 1 , wherein the interference image adds a color to the real object in the overlap region.

6. The method of claim 1 , wherein displaying, along with the direct light, the three-dimensional augmented scene comprises modifying the virtual image data based at least in part on the characteristic of the real object in the overlap region.

7. The method of claim 6 , wherein the modifying the virtual image data comprises subtracting color data from the virtual object in the overlap region.

8. The method of claim 1 , further comprising:

capturing the first image data of the real three-dimensional scene with at least the capturing device that is affixed relative to a head of the user,

determining the overlap region at least by determining a spatial overlap between a corresponding real object in the first image data that has been captured and the virtual object in the virtual image data; and

determining the first contrast characteristic of the real object in the overlap region comprises determining a corresponding characteristic of the corresponding real object in the first image data in the overlap region.

9. The method of claim 8 , wherein each of the first image data and the virtual image data comprises a plurality of pixels, each of the plurality of pixels is defined by at least one value, and displaying, along with the direct light, the three-dimensional augmented scene comprises modifying the at least one value of a first pixel of the virtual object in the overlap region.

10. The method of claim 9 , wherein the each of the plurality of pixels has three values defining a color vector.

11. The method of claim 10 , wherein the color vector comprises a red value, a green value, and a blue value.

12. The method of claim 9 , wherein modifying the at least one value of the each of the plurality of pixels of the virtual object in the overlap region comprises reducing the at least one value of the first pixel of the virtual object.

13. The method of claim 12 , wherein the at least one value of the first pixel of the virtual object is reduced by at least one corresponding value of a corresponding pixel of the real object.

14. The method of claim 13 , wherein the at least one value of the first pixel of the virtual object has three first values defining a first color vector, and at least one corresponding value of the corresponding pixel of the real object has three second values defining a second color vector.

15. A method of operating an augmented reality (AR) system, the method comprising:

capturing first image data from direct light in a three-dimensional scene with at least a capturing device that is affixed relative to a head of an user;

warping first image data into warped image data based at least in part a perspective of a user and a position of the capturing device;

determining an overlap region between a real object in the three-dimensional scene and a virtual object in virtual image data; and

presenting a three-dimensional augmented scene to the user at least by:

determining a characteristic of the real object in the overlap region;

identifying a pixel to be presented in the three-dimensional augmented scene to the user;

identifying real object data for displaying the characteristic of the real object at the pixel from the warped image data;

identifying virtual object data for displaying a corresponding portion of the virtual object at the pixel from the virtual image data;

determining pixel interference data for displaying the three-dimensional augmented scene at the pixel based at least in part upon the first data and the virtual object data;

displaying, along with the direct light, the three-dimensional augmented scene to the user at least by adding the real object data, the pixel interference data, and the virtual object data for the pixel so that that the characteristic of the real object appears to be completely obscured by the virtual object from the perspective of the user;

determining the overlap region at least by determining a spatial overlap between a corresponding real object in the first image data that has been captured and the virtual object in the virtual image data; and

determining the first contrast characteristic of the real object in the overlap region comprises determining a corresponding characteristic of the corresponding real object in the first image data in the overlap region, wherein

each of the first image data and the virtual image data comprises a plurality of pixels, wherein each of the plurality of pixels is defined by at least one value; and

displaying, along with the direct light, the three-dimensional augmented scene comprises:

selecting at least one value of a first pixel of a background for the virtual object in the overlap region;

generating interference data comprising a first plurality of pixels, wherein each of the first plurality of pixels is defined by at least one value; and

displaying the interference data as an interference image over the overlap region to the user such that the interference image is combined with the direct light to create the background for the virtual object in the overlap region.

16. The method of claim 15 , wherein each of the first plurality of pixels has three values defining a color vector.

17. The method of claim 15 , wherein the color vector comprises a red value, a green value, and a blue value.

18. The method of claim 15 , wherein displaying, along with the direct light, the three-dimensional augmented scene further comprises selecting at least one value of the first pixel of the background for the virtual object in the overlap region, wherein at least one corresponding value of a corresponding pixel of the interference data, when respectively added to the at least one value of the corresponding pixel of the corresponding real object in the image data in the overlap region, equals the at least one value for the pixel of the background for the virtual object in the overlap region.

19. The method of claim 18 , wherein displaying, along with the direct light, the three-dimensional augmented scene further comprises determining that at least one first value for another pixel of the corresponding real object in the first image data in the overlap region is below at least one second value of another corresponding pixel of the background for the virtual object in the overlap region, and modifying the at least one second value of another corresponding pixel of the virtual object in the overlap region.

20. The method of claim 15 , wherein all of the at least one value for the corresponding pixel of the corresponding real object in the first image data in the overlap region is respectively below the at least one value of pixel for the background for the virtual object in the overlap region.