IP Library Granted Patent US 10,535,199
Granted Patent B1
US 10,535,199 · App. 16/011,621 · Granted Jan 14, 2020

Systems and methods for determining a safety boundary for a mobile artificial reality user

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,535,199
App. No.
16/011,621
Granted
Jan 14, 2020
Kind
B1
Abstract

The disclosed method may include (1) sensing, via a depth-sensing subsystem, a plurality of locations in three-dimensional space corresponding to physical surfaces in a real-world environment, (2) determining a dominant plane within the real-world environment, (3) defining a three-dimensional grid that is aligned with the dominant plane, (4) identifying, based on the plurality of locations relative to the dominant plane, a set of grid coordinates within the three-dimensional grid that are indicative of the physical surfaces, and (5) determining, based on the set of grid coordinates, a safety boundary to be employed by a head-mounted display system to notify a user of the head-mounted display system of the user's proximity to the physical surfaces. Various other methods, systems, and computer-readable media are also disclosed.

Claims (59)

1. A method comprising:

sensing, via a depth-sensing subsystem, a plurality of locations in three-dimensional space corresponding to physical surfaces in a real-world environment;

determining a dominant plane within the real-world environment, wherein the dominant plane comprises a horizontal plane dividing the real-world environment into an upper region and a lower region;

defining a three-dimensional grid that is aligned with the dominant plane;

identifying, based on the plurality of locations relative to the dominant plane, a set of grid coordinates within the three-dimensional grid that are indicative of the physical surfaces;

defining, based on the set of grid coordinates, vertical columns of volumetric elements aligned with the three-dimensional grid to indicate the physical surfaces; and

determining, based on the vertical columns of volumetric elements, a safety boundary to be employed by a head-mounted display system to notify a user of the head-mounted display system of the user's proximity to the physical surfaces.

2. The method of claim 1 , wherein the dominant plane is located higher than a height of the user.

3. The method of claim 1 , wherein:

sensing the plurality of locations comprises sensing a first location; and

identifying the set of grid coordinates comprises:

defining a first volumetric element at the first location; and

defining one of more additional first volumetric elements contiguously with the first volumetric element to form a first vertical column of volumetric elements.

4. The method of claim 3 , wherein the one or more additional first volumetric elements extend vertically from the first volumetric element in a first direction away from the dominant plane.

5. The method of claim 4 , wherein:

sensing the plurality of locations comprises sensing a second location of the plurality of locations positioned vertically between the first volumetric element and the dominant plane; and

identifying the set of grid coordinates comprises:

translating the first volumetric element vertically to the second location; and

translating the one or more additional first volumetric elements vertically to maintain the first vertical column.

6. The method of claim 4 , wherein:

sensing the plurality of locations comprises sensing a second location of the plurality of locations positioned vertically from, and opposite the dominant plane from, the first location;

identifying the set of grid coordinates comprises:

defining a second volumetric element at the second location; and

defining one or more additional second volumetric elements contiguously with the second volumetric element to form a second vertical column of volumetric elements; and

the one or more additional second volumetric elements extend vertically from the second volumetric element in a second direction away from the dominant plane, opposite the first direction.

7. The method of claim 6 , wherein identifying the set of grid coordinates further comprises:

determining a distance between the first volumetric element and the second volumetric element; and

defining, based on the distance between the first volumetric element and the second volumetric element being less than a threshold value, one or more third volumetric elements between the first volumetric element and the second volumetric element to form a vertical column comprising the first vertical column, the second vertical column, and the third volumetric elements.

8. The method of claim 1 , wherein:

sensing the plurality of locations comprises sensing at least one vertical surface in the real-world environment; and

defining the three-dimensional grid comprises aligning the three-dimensional grid based on an orientation of the vertical surface.

9. The method of claim 8 , wherein the vertical surface comprises a wall.

10. The method of claim 1 , wherein the head-mounted display system comprises the depth-sensing subsystem.

11. The method of claim 1 , wherein:

sensing the plurality of locations comprises sensing a horizontal surface; and

determining the dominant plane comprises defining the dominant plane as parallel to the horizontal surface.

12. The method of claim 11 , wherein the horizontal surface comprises at least one of a ceiling or a floor.

13. The method of claim 1 , wherein:

sensing the plurality of locations comprises sensing a vertical surface; and

determining the dominant plane comprises defining the dominant plane as orthogonal to the vertical surface.

14. The method of claim 13 , wherein the vertical surface comprises a wall.

15. The method of claim 1 , further comprising sensing, via an inertial measurement unit, a first horizontal plane, wherein determining the dominant plane comprises defining the dominant plane as parallel to the first horizontal plane.

16. A non-transitory, tangible computer-readable storage medium having instructions stored thereon that, when executed by a processing system, cause the processing system to perform operations comprising:

receiving, from a depth-sensing subsystem, data indicating a plurality of locations in three-dimensional space corresponding to physical surfaces in a real-world environment;

determining a dominant plane within the real-world environment, wherein the dominant plane comprises a horizontal plane dividing the real-world environment into an upper region and a lower region;

defining a three-dimensional grid that is aligned with the dominant plane;

identifying, based on the plurality of locations relative to the dominant plane, a set of grid coordinates within the three-dimensional grid that are indicative of the physical surfaces;

defining, based on the set of grid coordinates, vertical columns of volumetric elements aligned with the three-dimensional grid to indicate the physical surfaces; and

determining, based on the vertical columns of volumetric elements, a safety boundary to be employed by a head-mounted display system to notify a user of the head-mounted display system of the user's proximity to the physical surfaces.

17. A head-mounted display system comprising:

a display device dimensioned to be secured to a head of a user in a manner that at least partially obscures visibility of a real-world environment to the user;

a depth-sensing subsystem that senses a plurality of locations in three-dimensional space corresponding to physical surfaces in the real-world environment; and

a processing system that:

determines a dominant plane within the real-world environment, wherein the dominant plane comprises a horizontal plane dividing the real-world environment into an upper region and a lower region;

defines a three-dimensional grid that is aligned with the dominant plane;

identifies, based on the plurality of locations relative to the dominant plane, a set of grid coordinates within the three-dimensional grid that are indicative of the physical surfaces;

defines, based on the set of grid coordinates, vertical columns of volumetric elements aligned with the three-dimensional grid to indicate the physical surfaces;

determines, based on the vertical columns of volumetric elements, a safety boundary associated with the physical surfaces; and

presents, to the user based on the safety boundary, an indication of the user's proximity to the physical surfaces.

Assignments (3)
CHANGE OF NAME Recorded May 26, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060199/0876 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 16, 2019
From: BOND, LARS ANDERS; KANTOR, NIV; GROSSINGER, NADAV
To: OCULUS VR, LLC,
Reel/Frame 048357/0686 →
CHANGE OF NAME Recorded Sep 20, 2018
From: OCULUS VR, LLC
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 047112/0836 →