IP Library Granted Patent US 11,734,867
Granted Patent B2
US 11,734,867 · App. 17/479,489 · Granted Aug 22, 2023

Detecting physical boundaries

Inventors: Seyedkoosha Mirhosseini (Port Jefferson, NY); Avi Bar-Zeev (Oakland, CA); Duncan A. K. McRoberts (Santa Clara, CA)
Assignee: Apple Inc.
G06T11/60G06F3/011G06T7/246G06T7/292
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,734,867
App. No.
17/479,489
Granted
Aug 22, 2023
Kind
B2
Abstract

Techniques for alerting a user, who is immersed in a virtual reality environment, to physical obstacles in their physical environment are disclosed.

Claims (71)

1. An electronic device, comprising:

one or more cameras;

a memory;

one or more processors operatively coupled to the memory; and

computer-executable instructions stored in the memory and configured to be executed by the one or more processors, the computer-executable instructions including instructions for:

displaying a virtual reality environment;

determining a distance between the electronic device and a first physical object;

determining a first type of the first physical object;

determining whether the distance between the electronic device and the first physical object is within a first threshold distance, wherein the first threshold distance is based on the determined first type of the first physical object;

in accordance with a determination that the distance is within the first threshold distance, displaying a visual effect in the virtual reality environment,

determining whether the distance between the electronic device and the first physical object is within a second threshold distance;

in accordance with a determination that the distance is within the second threshold distance, displaying a visual representation of at least part of a physical environment, the visual representation provided using the one or more cameras;

determining a second distance between the electronic device and a second physical object;

determining a second type of the second physical object, wherein the first type of the first physical object is different from the second type of the second physical object;

determining whether the second distance between the electronic device and the second physical object is within a third threshold distance, wherein the third threshold distance is based on the determined second type of the second physical object, and wherein first threshold distance is different from the third threshold distance;

in accordance with a determination that the second distance is within the third threshold distance, displaying the visual effect in the virtual reality environment;

determining whether the second distance between the electronic device and the second physical object is within a fourth threshold distance; and

in accordance with a determination that the second distance is within the fourth threshold distance, displaying a second visual representation of at least part of the physical environment, the second visual representation provided using the one or more cameras.

2. The electronic device of claim 1 , wherein the second threshold distance is based on the type of the first physical object.

3. The electronic device of claim 1 , wherein the type of the first physical object is associated with a risk level of the first physical object.

4. The electronic device of claim 1 , wherein the electronic device is a wearable headset.

5. The electronic device of claim 1 , wherein determining whether the distance between the electronic device and the first physical object is within the first threshold distance comprises:

determining whether the electronic device is moving toward the first physical object in a physical environment.

6. The electronic device of claim 1 , wherein at least one of the first threshold distance and the second threshold distance is based on speed of movement of the electronic device.

7. The electronic device of claim 1 , wherein the computer-executable instructions further include instructions for:

determining the first threshold distance and the second threshold distance based on the first physical object.

8. The electronic device of claim 1 , wherein displaying the visual effect comprises:

displaying one or more virtual objects in the virtual reality environment, wherein the one or more virtual objects is an indication that the first physical object is within the first threshold distance from the electronic device.

9. The electronic device of claim 8 , wherein the computer-executable instructions further include instructions for:

varying an appearance of at least one of the displayed one or more virtual objects based on the distance between the electronic device and the first physical object.

10. The electronic device of claim 1 , wherein displaying the virtual reality environment comprises displaying a surface representing a ground surface, and wherein displaying the visual effect comprises:

displaying a darkened area on the ground surface, wherein the darkened area corresponds to a location of the first physical object in a physical environment relative to the electronic device.

11. The electronic device of claim 10 , wherein the computer-executable instructions further include instructions for:

determining whether the electronic device is moving towards or moving away from the first physical object; and

in accordance with a determination that the electronic device is moving towards the first physical object, darkening the darkened area based on the distance between the electronic device and the first physical object.

12. The electronic device of claim 11 , wherein the computer-executable instructions further include instructions for:

in accordance with a determination that the electronic device is moving away from the first physical object, lightening the darkened area.

13. The electronic device of claim 10 , wherein a dimension of the darkened area corresponds to a size of the first physical object.

14. The electronic device of claim 1 , wherein determining whether the distance between the electronic device and the first physical object in the physical environment is within the second threshold distance comprises:

determining whether the electronic device is moving toward the first physical object; and

in accordance with a determination that the electronic device is moving toward the first physical object, determining whether the distance between the electronic device and the first physical object of the physical environment is less than or equal to the second threshold distance.

15. The electronic device of claim 1 , wherein the visual effect corresponds to the distance associated with the first physical object.

16. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device with one or more cameras, the one or more programs including instructions for:

displaying a virtual reality environment;

determining a distance between the electronic device and a first physical object;

determining a first type of the first physical object;

determining whether the distance between the electronic device and the first physical object is within a first threshold distance, wherein the first threshold distance is based on the determined first type of the first physical object;

in accordance with a determination that the distance is within the first threshold distance, displaying a visual effect in the virtual reality environment;

determining whether the distance between the electronic device and the first physical object is within a second threshold distance;

in accordance with a determination that the distance is within the second threshold distance, displaying a visual representation of at least part of a physical environment, the visual representation provided using the one or more cameras;

determining a second distance between the electronic device and a second physical object;

determining a second type of the second physical object, wherein the first type of the first physical object is different from the second type of the second physical object;

determining whether the second distance between the electronic device and the second physical object is within a third threshold distance, wherein the third threshold distance is based on the determined second type of the second physical object, and wherein first threshold distance is different from the third threshold distance;

in accordance with a determination that the second distance is within the third threshold distance, displaying the visual effect in the virtual reality environment;

determining whether the second distance between the electronic device and the second physical object is within a fourth threshold distance; and

in accordance with a determination that the second distance is within the fourth threshold distance, displaying a second visual representation of at least part of the physical environment, the second visual representation provided using the one or more cameras.

17. A method for indicating physical obstacles while displaying a virtual reality environment, the method comprising:

at an electronic device with one or more processors, memory, and one or more cameras:

displaying a virtual reality environment;

determining a distance between the electronic device and a first physical object;

determining a first type of the first physical object;

determining whether the distance between the electronic device and the first physical object is within a first threshold distance, wherein the first threshold distance is based on the determined first type of the first physical object;

in accordance with a determination that the distance is within the first threshold distance displaying a visual effect in the virtual reality environment;

determining whether the distance between the electronic device and the first physical object is within a second threshold distance;

in accordance with a determination that the distance is within the second threshold distance, displaying a visual representation of at least part of a physical environment, the visual representation provided using the one or more cameras;

determining a second distance between the electronic device and a second physical object;

determining a second type of the second physical object, wherein the first type of the first physical object is different from the second type of the second physical object;

determining whether the second distance between the electronic device and the second physical object is within a third threshold distance, wherein the third threshold distance is based on the determined second type of the second physical object, and wherein first threshold distance is different from the third threshold distance;

in accordance with a determination that the second distance is within the third threshold distance, displaying the visual effect in the virtual reality environment;

determining whether the second distance between the electronic device and the second physical object is within a fourth threshold distance; and

in accordance with a determination that the second distance is within the fourth threshold distance, displaying a second visual representation of at least part of the physical environment, the second visual representation provided using the one or more cameras.

Continuity (5)
Continuation 16833395 · Mar 27, 2020
Continuation PCTUS2018052719 · Sep 25, 2018
Provisional Application 62631048 · Feb 15, 2018
Provisional Application 62566306 · Sep 29, 2017
Related Publication 20220108508A1 · Apr 7, 2022
Cited By (3)
US 12,195,006 US 12,366,914 US 12,608,169