IP Library Granted Patent US 9,552,676
Granted Patent B2
US 9,552,676 · App. 15/132,852 · Granted Jan 24, 2017

Wearable computer with nearby object response

Inventors: Adrian Wong (Mountain View, CA); Xiaoyu Miao (Mountain View, CA)
Assignee: Google Inc.
G06T19/006G02B27/0093G02B27/017G02B27/0172G02B27/0176G06F3/013G06F3/0482G06F3/167G06K9/00664G02B2027/014G02B2027/015G02B2027/0138G02B2027/0178G06T2200/24
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,552,676
App. No.
15/132,852
Granted
Jan 24, 2017
Kind
B2
Abstract

Exemplary methods and systems relate to detecting physical objects near a substantially transparent head-mounted display (HMD) system and activating a collision-avoidance action to alert a user of the detected objects. Detection techniques may include receiving data from distance and/or relative movement sensors and using this data as a basis for determining an appropriate collision-avoidance action. Exemplary collision-avoidance actions may include de-emphasizing virtual objects displayed on the HMD to provide a less cluttered view of the physical objects through the substantially transparent display and/or presenting new virtual objects.

Claims (59)

1. A computer-implemented method comprising:

causing a head-mountable device (HMD) to display a field of view to a user-interface mapped to a sphere, wherein the field of view is moveable with respect to the user-interface, and wherein the user-interface comprises: (i) a view region, and (ii) a set of selectable content objects arranged in at least a partial ring;

associating a forward-looking head position with a forward-looking location of the field of view on the sphere, wherein a default arrangement of the spherical user-interface both: (i) locates the view region within the forward-looking field of view, and (ii) locates the set of selectable content objects substantially outside the forward-looking field of view, wherein the forward-looking field of view is in a direction that is generally parallel to the ground;

displaying one or more virtual objects in the view region;

using data from one or more first sensors as a basis for determining a distance between the HMD and a physical object; and

using the determined distance between the HMD and the physical object as basis for initiating a collision-avoidance action, wherein the collision-avoidance action de-emphasizes one or more of the displayed virtual objects to provide a less-obstructed view of the physical object.

2. The method of claim 1 , wherein using the determined distance as basis for initiating the collision-avoidance action comprises:

determining that the physical object is less than a threshold distance from the HMD; and responsively initiating the collision-avoidance action.

3. The method of claim 1 , further comprising:

using data from one or more second sensors to determine a relative movement of the physical object with respect to the HMD; and

using the relative movement as a further basis for initiating the collision-avoidance action.

4. The method of claim 3 , wherein using the relative movement as a further basis for initiating the collision-avoidance action comprises:

using the relative movement as a basis for determining the threshold distance between the physical object and the HMD; and

determining that the physical object is less than the threshold distance from the HMD and responsively initiating the collision-avoidance action.

5. The method of claim 3 , wherein using the relative movement as a further basis for initiating the collision-avoidance action comprises:

using (a) the distance between the physical object and the HMD and (b) the relative movement to determine a time period until collision; and

determining that the time period until collision is less than a threshold time period and responsively initiating the collision-avoidance action.

6. The method of claim 3 , further comprising, subsequent to initiating the collision-avoidance action:

using data from the one or more second sensors as a basis for determining a subsequent relative movement of the physical object with respect to the HMD;

determining that a direction of the subsequent relative movement of the physical object is directed sufficiently away from the HMD such that collision with the physical object is unlikely; and

in response to determining that the subsequent relative movement of the physical object is directed sufficiently away from the HMD, re-displaying in the view region the at least one virtual object that was de-emphasized in the collision-avoidance action.

7. The method of claim 1 , further comprising:

using data from one or more third sensors to determine that the HMD is in motion; and

using the determination that the HMD is in motion as a further basis for initiating a collision-avoidance action.

8. The method of claim 1 , further comprising:

determining that the view region is sufficiently cluttered; and

using the determination that the view region is sufficiently cluttered as a further basis for initiating the collision-avoidance action.

9. The method of claim 1 , wherein de-emphasizing at least one of the virtual objects comprises removing at least one of the virtual objects from the view region.

10. The method of claim 1 , wherein the moving the at least one of said virtual objects comprises efficiently portraying a potential hazard by moving the at least one of said virtual objects quickly from a center point of the view region toward outer edges of the view region.

11. The method of claim 10 , further comprising:

determining a portion of the view region that overlays the physical object;

wherein the at least one of said virtual objects is moved quickly away from the portion of the view region that overlays the physical object.

12. The method of claim 1 , wherein the physical object is outside of the field of view provided in the see-through display, and wherein the collision-avoidance action further comprises displaying a visual indication of a direction of the physical object.

13. The method of claim 1 , wherein the collision-avoidance action further comprises presenting an audio alert.

14. The method of claim 1 , wherein the collision-avoidance action further comprises displaying a visual indication that highlights the physical object.

15. A computing system comprising:

at least one processor;

one or more first sensors operable to generate data indicative of distances to physical objects in an environment of a head-mountable device (HMD);

data storage comprising logic executable by the at least one processor to:

cause a display of the HMD to display a field of view to a user-interface mapped to a sphere, wherein the field of view is moveable with respect to the user-interface, and wherein the user-interface comprises: (i) a view region, and (ii) a set of selectable content objects arranged in at least a partial ring;

associate a forward-looking head position with a forward-looking location of the field of view on the sphere, wherein a default arrangement of the spherical user-interface both: (i) locates the view region within the forward-looking field of view, and (ii) locates the set of selectable content objects substantially outside the forward-looking field of view, wherein the forward-looking field of view is in a direction that is generally parallel to the ground;

display one or more virtual objects in the view region;

use data from one or more first sensors as a basis for determining a distance between the HMD and a physical object; and

use the determined distance between the HMD and the physical object as basis to initiate a collision-avoidance action, wherein the collision-avoidance action de-emphasizes one or more of the displayed virtual objects to provide a less-obstructed view of the physical object.

16. The computing system of claim 15 , wherein the user-input comprises detection by sensors that the head-mounted display has moved in specified movement pattern.

17. A non-transitory computer-readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:

causing a head-mountable device (HMD) to display a field of view to a user-interface mapped to a sphere, wherein the field of view is moveable with respect to the user-interface, and wherein the user-interface comprises: (i) a view region, and (ii) a set of selectable content objects arranged in at least a partial ring;

associating a forward-looking head position with a forward-looking location of the field of view on the sphere, wherein a default arrangement of the spherical user-interface both: (i) locates the view region within the forward-looking field of view, and (ii) locates the set of selectable content objects substantially outside the forward-looking field of view, wherein the forward-looking field of view is in a direction that is generally parallel to the ground;

displaying one or more virtual objects in the view region;

using data from one or more first sensors as a basis for determining a distance between the HMD and a physical object; and

using the determined distance between the HMD and the physical object as basis for initiating a collision-avoidance action, wherein the collision-avoidance action de-emphasizes one or more of the displayed virtual objects to provide a less-obstructed view of the physical object.

18. The non-transitory computer-readable medium of claim 17 , wherein the functions further comprise:

using data from one or more second sensors to determine a relative movement of the physical object with respect to the HMD; and

using the relative movement as a further basis for initiating the collision-avoidance action.

19. The non-transitory computer-readable medium of claim 18 , wherein the functions further comprise subsequent to initiating the collision-avoidance action:

using data from the one or more second sensors as a basis for determining a subsequent relative movement of the physical object with respect to the HMD;

determining that a direction of the subsequent relative movement of the physical object is directed sufficiently away from the HMD such that collision with the physical object is unlikely; and

in response to determining that the subsequent relative movement of the physical object is directed sufficiently away from the HMD, re-displaying in the view region the at least one virtual object that was de-emphasized in the collision-avoidance action.

20. The non-transitory computer-readable medium of claim 17 , wherein the moving the at least one of said virtual objects comprises efficiently portraying a potential hazard by moving the at least one of said virtual objects quickly from a center point of the view region toward outer edges of the view region.

Assignments (2)
CHANGE OF NAME Recorded Oct 2, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044097/0658 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 19, 2016
From: WONG, ADRIAN; MIAO, XIAOYU
To: GOOGLE INC.
Reel/Frame 038323/0459 →
Continuity (3)
Continuation 14737745 · Jun 12, 2015
Continuation 13267963 · Oct 7, 2011
Related Publication 20160232717A1 · Aug 11, 2016