IP Library Granted Patent US 10,044,925
Granted Patent B2
US 10,044,925 · App. 15/240,850 · Granted Aug 7, 2018

Techniques for setting focus in mixed reality applications

Inventors: Hang Li (Bellevue, WA); Vishal Jain (Redmond, WA); Maria Cheng Lei (Bellevue, WA); William Douglas Guyman (Medina, WA)
Assignee: Microsoft Technology Licensing, LLC
H04N5/23212G06T7/0051G06T19/006H04N5/2254H04N5/23219H04N5/23293
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,044,925
App. No.
15/240,850
Granted
Aug 7, 2018
Kind
B2
Abstract

Described are examples for capturing one or more real world images for display with one or more computer-generated images (e.g., holograms). One or more computer-generated images for overlaying over one or more real world images can be received, and a depth for overlaying at least one of the computer-generated images over the one or more real world images can be determined. A lens of a camera can be focused based on this depth, and the one or more real world images can be captured via the camera with the lens focused based on the depth. The one or more real world images can be provided as one or more mixed reality images with the one or more computer-generated images overlaid on the one or more real world images, such that the image is focused on objects near the one or more computer-generated images.

Claims (38)

1. A computing device, comprising:

a camera comprising a lens configured to capture and provide one or more real world images; and

at least one processor configured to determine whether to transform the one or more real world images to a mixed reality image;

wherein when the at least one processor determines not to transform the one or more real world images, the at least one processor is configured to perform one or more auto-focusing processes of the camera to focus the one or more real world images; and

wherein when the at least one processor determines to transform the one or more real world images, the at least one processor is configured to:

overlay one or more computer-generated images over the one or more real world images; and

override, based on determining to transform the one or more real world images, the one or more auto-focusing processes of the camera by causing the camera to focus the lens for capturing the one or more real world images based at least in part on a depth at which the at least one processor overlays at least one of the one or more computer-generated images.

2. The computing device of claim 1 , wherein the at least one processor is further configured to cause the camera to focus the lens at the depth of the at least one of the one or more computer-generated images.

3. The computing device of claim 1 , wherein the at least one processor is configured to determine a selection of the at least one computer-generated image based on a size of the at least one computer-generated image, and wherein the at least one processor is configured to cause the camera to focus the lens at least in part in response to the selection.

4. The computing device of claim 1 , wherein the at least one processor is configured to cause the camera to focus the lens at the depth corresponding to a point of the at least one of the one or more computer-generated images having a level of detail that achieves a threshold.

5. The computing device of claim 1 , wherein the at least one processor is configured to cause the camera to focus the lens at the depth corresponding to a size or distance from the computing device of the at least one of the one or more computer-generated images.

6. The computing device of claim 1 , wherein the at least one processor is configured to receive a selection of the at least one of the one or more computer-generated images, and wherein the at least one processor is configured to cause the camera to focus the lens at least in part in response to the selection.

7. The computing device of claim 6 , wherein the selection corresponds to a detected gesture to select the at least one of the one or more computer-generated images presented on an interface on a display.

8. The computing device of claim 6 , further comprising a gaze tracking sensor configured to track a position of one or more eyes of a user, wherein the selection corresponds to a determination that the one or more eyes of the user are positioned to view the at least one of the one or more computer-generated images.

9. The computing device of claim 6 , wherein the selection corresponds to determining one or more heuristics of the at least one of the one or more computer-generated images.

10. The computing device of claim 1 , further comprising a display for displaying the one or more real world images overlaid with the one or more computer-generated images, wherein the display is at least one of positioned on the computing device or remotely located from the computing device.

11. A method for capturing one or more real world images for providing with one or more computer-generated images, comprising:

receiving one or more computer-generated images for overlaying over one or more real world images;

determining a depth for overlaying at least one of the one or more computer-generated images over the one or more real world images;

overriding, based on receiving the one or more computer-generated images, an auto-focusing process of a camera at least in part by instead focusing a lens of a camera based on the depth;

capturing the one or more real world images via the camera with the lens focused based on the depth; and

providing the one or more real world images as one or more mixed reality images with the one or more computer-generated images overlaid on the one or more real world images.

12. The method of claim 11 , wherein focusing the lens further comprises focusing the lens at the depth for overlaying the at least one of the one or more computer-generated images.

13. The method of claim 11 , further comprising determining a selection of the at least one computer-generate image based on a size of the at least one computer-generated image, and wherein focusing the lens is based at least in part on the selection.

14. The method of claim 11 , wherein focusing the lens further comprises focusing the lens at the depth corresponding to a point of the at least one of the one or more computer-generated images having a level of detail that achieves a threshold.

15. The method of claim 11 , further comprising receiving a selection of the at least one of the one or more computer-generated images, and wherein focusing the lens is based at least in part on the selection.

16. The method of claim 15 , further comprising detecting a gesture to select the at least one of the one or more computer-generated images presented on an interface on the display, wherein the selection corresponds to the gesture.

17. The method of claim 15 , further comprising:

tracking position of one or more eyes of a user; and

determining that the one or more eyes of the user are positioned to view the at least one of the one or more computer-generated images, wherein the selection corresponds to the determination.

18. A non-transitory computer-readable medium comprising code for capturing one or more real world images for providing with one or more computer-generated images, the code comprising:

code for receiving one or more computer-generated images for overlaying on one or more real world images;

code for determining a depth for overlaying at least one of the one or more computer-generated images on the one or more real world images;

code for overriding, based on receiving the one or more computer-generate images, an auto-focusing process of a camera at least in part by instead focusing a lens of a camera based on the depth;

code for capturing the one or more real world images via the camera with the lens focused based on the depth; and

code for providing the one or more real world images as one or more mixed reality images with the one or more computer-generated images overlaid on the one or more real world images.

19. The non-transitory computer-readable medium of claim 18 , wherein the code for focusing the lens focuses the lens at the depth for overlaying the at least one of the one or more computer-generated images.

20. The non-transitory computer-readable medium of claim 18 , wherein the code further comprises code for determining a selection of the at least one computer-generate image based on a size of the at least one computer-generated image, and wherein focusing the lens is based at least in part on the selection.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 8, 2016
From: LI, HANG; JAIN, VISHAL; LEI, MARIA CHENG; GUYMAN, WILLIAM DOUGLAS
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 039676/0810 →
Continuity (1)
Related Publication 20180054562A1 · Feb 22, 2018