IP Library › Granted Patent US 11,620,761
Granted Patent B2
US 11,620,761 · App. 17/341,218 · Granted Apr 4, 2023

Depth sensing via device case

Inventors: Navid Poulad (San Jose, CA); Rayna Demaster-Smith (Kirkland, WA)
Assignee: Microsoft Technology Licensing, LLC
G06T7/593G02B27/0172G06T7/11G06T7/40G06T7/521G06T7/85G06T15/04G06T17/00H04N9/045H04N13/239H04N13/344G02B2027/014G02B2027/0174G02B2027/0178G06T2207/10024G06T2207/10028G06T2207/30196H04N2013/0081H04S7/301
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,620,761
App. No.
17/341,218
Granted
Apr 4, 2023
Kind
B2
Abstract

Examples are disclosed that relate to displaying a hologram via an HMD. One disclosed example provides a method comprising obtaining depth data from a direct-measurement depth sensor included in the case for the HMD, the depth data comprising a depth map of a real-world environment. The method further comprises determining a distance from the HMD to an object in the real-world environment using the depth map, obtaining holographic imagery for display based at least upon the distance, and outputting the holographic imagery for display on the HMD.

Claims (51)

1. On a computing system comprising a head-mounted display device (HMD) and a case for the head-mounted display device, a method for displaying a hologram via the HMD, the method comprising:

obtaining depth data from a direct-measurement depth sensor included in the case for the HMD, the depth data comprising a depth map of a real-world environment;

sending the depth map from the case to the HMD;

on the HMD, determining a distance from the HMD to an object in the real-world environment using the depth map;

obtaining holographic imagery for display based at least upon the distance; and

outputting the holographic imagery for display on the HMD.

2. The method of claim 1 , wherein obtaining the depth data from the direct-measurement depth sensor comprises obtaining the depth data from a time-of-flight sensor on the case.

3. The method of claim 1 , further comprising obtaining a depth image of a user of the HMD via the direct-measurement depth sensor during a holographic communication session, sending the depth image of the user to the HMD during the holographic communication session, and displaying on the HMD a representation of the user based upon the depth image of the user.

4. The method of claim 3 , further comprising obtaining texture data representing an appearance of the user of the HMD, and displaying the texture data in the representation of the user.

5. The method of claim 4 , further comprising sending the depth image of the user to another HMD participating in the holographic communication session.

6. The method of claim 1 , wherein the depth data further comprises a depth image of a plurality of people including a user of the HMD, the method further comprising segmenting each person of the plurality of people.

7. The method of claim 1 , wherein the depth data comprises first depth data from a first direct-measurement depth sensor, wherein the depth map comprises a first depth map, and the method further comprising:

obtaining second depth data from a second direct-measurement depth sensor, the second depth data comprising a second depth map of the real-world environment;

detecting an overlapping region in the first depth map and the second depth map; and

using the overlapping region to combine the first depth map and the second depth map.

8. The method of claim 1 , wherein the HMD includes a depth imaging system comprising a stereo camera arrangement configured to obtain indirect-measurement depth data, wherein the distance is a first determined distance, and wherein the method further comprises calibrating the depth imaging system by

obtaining indirect-measurement depth data for the real-world environment via the depth imaging system of the HMD;

determining a second determined distance from the HMD to the object in the real-world environment using the indirect-measurement depth data;

comparing the first determined distance and the second determined distance to determine a correction for the indirect-measurement depth data; and

applying the correction to subsequently measured indirect-measurement depth data.

9. The method of claim 1 , further comprising obtaining a three-dimensional model of a target object by scanning the target object with the direct-measurement depth sensor included in the case from a plurality of angles.

10. The method of claim 1 , further comprising using the depth map to determine a placement of the holographic imagery.

11. A system comprising:

a head-mounted display device (HMD) and a case for the head-mounted display device,

the HMD comprising

a see-through display system,

a first communications system,

an indirect-measurement depth sensing system, and

a first computing system configured to control the display of images via the see-through display system and to control communication with the case via the first communications system; and

the case comprising

a direct-measurement depth sensor,

a second communications system, and

a second computing system configured to control the direct-measurement depth sensor to acquire depth data of a real-world environment and to control the second communications system to send depth data to the HMD.

12. The system of claim 11 , wherein the direct-measurement depth sensor comprises a time-of-flight sensor.

13. The system of claim 11 , wherein the case further comprises a red/green/blue (RGB) intensity image sensor.

14. The system of claim 13 , wherein one or more of the first computing system and the second computing system comprises instructions executable to use RGB image data acquired via the RGB intensity image sensor as texture data for corresponding depth data.

15. The system of claim 11 , wherein one or more of the first computing system and the second computing system comprises instructions executable to calibrate the indirect-measurement depth sensing system using depth data from the direct-measurement depth sensor.

16. The system of claim 11 , wherein one or more of the first computing system and the second computing system comprises instructions executable to determine a distance from the HMD to an object in the real-world environment using the depth data of the real-world environment, and to compute a holographic image for display based upon the distance.

17. The system of claim 11 , wherein the depth data further comprises a three-dimensional model of an object in the real-world environment.

18. The system of claim 11 , wherein the case is operatively configured to obtain a depth image of a user of the HMD via the direct-measurement depth sensor during a holographic communication session.

19. A system comprising:

a head-mounted display device (HMD) and a case for the head-mounted display device,

the HMD comprising a see-through display system, a first communications system, and a first computing system,

the case comprising a direct-measurement depth sensor, a second communications system, and a second computing system, and

the system also comprising a microphone located on the case or the HMD, wherein the system is configured to conduct a holographic communication session by

acquiring a depth image of a user of the HMD via the direct-measurement depth sensor on the case,

acquiring first acoustic data capturing a voice of the user of the HMD via the microphone,

sending the depth image of the user of the HMD and the first acoustic data to another computing device,

receiving second acoustic data and image data from the other computing device, and

presenting the second acoustic data and image data received from the other computing device.

20. The system of claim 19 , wherein the case further comprises a red/green/blue (RGB) image sensor, and wherein the system is further configured to obtain texture data representing an appearance of the user of the HMD via the RGB image sensor, and to send the texture data to the other computing device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 7, 2021
From: POULAD, NAVID; DEMASTER-SMITH, RAYNA
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 056460/0197 →
Continuity (1)
Related Publication 20220392093A1 · Dec 8, 2022