IP Library Granted Patent US 11,435,593
Granted Patent B1
US 11,435,593 · App. 16/403,646 · Granted Sep 6, 2022

Systems and methods for selectively augmenting artificial-reality experiences with views of real-world environments

Inventors: Sebastian Sztuk (Menlo Park, CA); Steven Paul Lansel (East Palo Alto, CA)
Assignee: Meta Platforms Technologies, LLC
G02B27/148G02B6/28G02B27/0172G06T7/70G02B2027/014G02B2027/0138G06T19/006
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,435,593
App. No.
16/403,646
Granted
Sep 6, 2022
Kind
B1
Abstract

The disclosed computer-implemented method may include (1) displaying one or more images to a user via a display comprising multiple display regions, (2) switching each of the display regions to a blocking state in which a view of the user's real-world environment in a corresponding region of the user's field of view is blocked from the user, (3) detecting a pass-through triggering event involving one or more objects in the user's real-world environment, (4) identifying one or more display regions corresponding to a region of the user's field of view occupied by the object, and (5) switching each of the one or more display regions to a pass-through state in which the view of the user's real-world environment in the corresponding region of the user's field of view is passed through to the user. Various other methods, systems, and computer-readable media are also disclosed.

Claims (47)

1. A computer-implemented method comprising:

displaying one or more images to a user via a pass-through display comprising multiple display regions;

switching each of the display regions to a blocking state in which a view of the user's real-world environment in a corresponding region of the user's field of view is blocked from the user;

detecting a pass-through triggering event involving a real object in the user's real-world environment;

identifying, in response to the pass-through triggering event, display regions of the pass-through display that block, when in the blocking state, a region of the user's field of view occupied by the real object; and

enabling the user to see the real object contemporaneous with the one or more images by switching each of the display regions that block the region of the user's field of view occupied by the real object to a pass-through state while the one or more images are displayed to the user via display regions of the pass-through display that do not block, when in the blocking state, the region of the user's field of view occupied by the real object.

2. The computer-implemented method of claim 1 , wherein:

the pass-through display is a head-mounted display;

the one or more images convey an evolving artificial-reality environment;

the pass-through triggering event is detected while all the display regions of the pass-through display are in the blocking state and the user is immersed in the evolving artificial-reality environment.

3. The computer-implemented method of claim 2 , wherein:

the real object is a type of real object with which the user can interact with the evolving artificial-reality environment; and

detecting the pass-through triggering event comprises detecting when the real object is in the user's field of view.

4. The computer-implemented method of claim 1 , wherein:

the real object comprises a hazard in the user's real-world environment; and

detecting the pass-through triggering event comprises detecting when the hazard endangers the user above a predetermined threshold.

5. The computer-implemented method of claim 1 , wherein:

the real object is a type of real object with which the user is likely to interact; and

the pass-through triggering event is known to precede an interaction by the user with the real object.

6. The computer-implemented method of claim 1 , wherein:

the real object is a type of real object with which the user is likely to interact; and

detecting the pass-through triggering event comprises detecting an action by the real object that is likely to precede an interaction by the user with the real object.

7. The computer-implemented method of claim 1 , wherein:

the real object is a type of real object with which the user is likely to interact; and

detecting the pass-through triggering event comprises detecting an action by the user that indicates an intention to interact with the real object.

8. The computer-implemented method of claim 7 , wherein detecting the action that indicates the intention to interact with the real object comprises detecting when the user turns towards the real object.

9. The computer-implemented method of claim 7 , wherein detecting the action that indicates the intention to interact with the real object comprises detecting when the user reaches towards the real object.

10. The computer-implemented method of claim 1 , wherein:

the real object comprises a person; and

detecting the pass-through triggering event comprises detecting an action by the person that is likely to precede an interaction by the user with the person.

11. The computer-implemented method of claim 1 , wherein:

the real object comprises a person; and

detecting the pass-through triggering event comprises detecting an action by the user that indicates an intention to interact with the person.

12. The computer-implemented method of claim 1 , wherein:

the real object comprises a portion of the user's body; and

detecting the pass-through triggering event comprises detecting when the portion of the user's body is in the user's field of view.

13. The computer-implemented method of claim 1 , wherein:

the real object comprises a portion of the user's body; and

detecting the pass-through triggering event comprises detecting an action by the user that indicates an intention to use the portion of the user's body to interact with the user's real-world environment.

14. The computer-implemented method of claim 1 , further comprising:

detecting a movement of the real object relative to the field of view of the user;

identifying, after the movement, additional display regions of the pass-through display that block the region of the user's field of view occupied by the real object; and

switching each of the additional display regions to the pass-through state.

15. The computer-implemented method of claim 1 , further comprising:

detecting a movement of the real object relative to the field of view of the user;

identifying, after the movement, at least one of the display regions of the pass-through display that no longer block the region of the user's field of view occupied by the real object; and

returning the at least one of the display regions to the blocking state.

Assignments (2)
CHANGE OF NAME Recorded Jun 7, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060291/0526 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 24, 2019
From: SZTUK, SEBASTIAN; LANSEL, STEVEN PAUL
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 049277/0063 →
Cited By (7)
US 12,223,104 US 12,393,031 US 12,444,217 US 12,512,024 US 12,524,072 US 12,531,972 US 12,684,106