IP Library Granted Patent US 10,379,346
Granted Patent B2
US 10,379,346 · App. 15/713,995 · Granted Aug 13, 2019

Methods and devices for rendering interactions between virtual and physical objects on a substantially transparent display

Inventors: Adrian Wong (Mountain View, CA); Xiaoyu Miao (Sunnyvale, CA)
Assignee: Google LLC
G02B27/017G06T19/006G02B2027/014G02B2027/0138G02B2027/0141G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,379,346
App. No.
15/713,995
Granted
Aug 13, 2019
Kind
B2
Abstract

Disclosed are methods and devices for rendering interactions between virtual and physical objects on a substantially transparent display are disclosed. In one embodiment, the method includes displaying a user-interface on a substantially transparent display of a wearable computing device. The method further includes displaying a virtual object in the view region at a focal length along a first line of sight and detecting a physical object at a physical distance along a second line of sight. The method still further includes determining that a relationship between the focal length and the physical distance is such that the virtual object and the physical object appear substantially co-located in a user-view through the view region and, responsive to the determination, initiating a collision action between the virtual object and the physical object.

Claims (53)

1. A computer-implemented method comprising:

displaying a user-interface on a display of a mobile computing device, wherein the user-interface comprises a view region via which virtual objects and real-world objects are viewable;

displaying a virtual object in the view region, wherein the virtual object is displayed at a focal length from the display along a first line of sight;

determining a physical distance, along the first line of sight, between the display and a physical object;

determining that the focal length is substantially equal to the physical distance to the physical object, such that the virtual object and the physical object appear substantially co-located, in the view region; and

responsive to the determination that the focal length is substantially equal to the physical distance, initiating a collision action between the virtual object and the physical object, wherein the collision action comprises a graphical modification of the virtual object, and wherein the graphical modification of the virtual object comprises the mobile computing device causing the virtual object to move away from the physical object.

2. The method of claim 1 , wherein the display comprises a see-through display.

3. The method of claim 2 , wherein the see-through display is a virtual see-through display.

4. The method of claim 1 , wherein determining the physical distance to the physical object comprises:

capturing a first image of the physical object; and

based on the first image, determining the physical distance to the physical object.

5. The method of claim 4 , further comprising:

capturing a second image of the physical object, wherein the first image is captured at a first time and the second image is captured at a second time; and

based on the first image, the second image, the first time, and the second time, determining a velocity of the physical object.

6. The method of claim 1 , wherein determining the physical distance to the physical object comprises:

emitting a series of pulsed signals;

detecting reflections of at least some of the pulsed signals; and

based on the reflections, detecting the physical object.

7. The method of claim 1 , wherein determining the physical distance to the physical object comprises:

emitting optical beams forming a three-dimensional lattice;

detecting disturbances in the lattice; and

based on the disturbances, detecting the physical object.

8. The method of claim 1 , further comprising:

using at least one sensor to detect a velocity of the physical object.

9. The method of claim 1 , wherein the graphical modification of the virtual object is based on a velocity of the physical object.

10. The method of claim 1 , wherein the graphical modification of the virtual object further comprises the virtual object changing shape.

11. The method of claim 1 , wherein the graphical modification of the virtual object further comprises moving the virtual object out of the view region.

12. The method of claim 1 , wherein the user-interface is initially configured such that a content region is substantially arranged along an at least partial ring above the view region.

13. A computing device comprising:

at least one processor; and

data storage comprising logic executable by the at least one processor to:

display a user-interface on a display, wherein the user-interface comprises a view region;

display a virtual object in the view region, wherein the virtual object is displayed at a focal length from the display along a first line of sight;

determine a physical distance, along a first line of sight, between the display and a physical object;

determine that the focal length is substantially equal to the physical distance to the physical object, such that the virtual object and the physical object appear substantially co-located in the view region; and

responsive to the determination that the focal length is substantially equal to the physical distance, initiate a collision action between the virtual object and the physical object, wherein the collision action comprises a graphical modification of the virtual object, and wherein the graphical modification of the virtual object comprises the computing device causing the virtual object to move away from the physical object.

14. The computing device of claim 13 , wherein the display comprises a see-through graphic display.

15. The computing device of claim 13 , wherein the see-through graphic display is a virtual see-through display.

16. The computing device of claim 13 , further comprising:

an emitter configured to emit a series of pulsed signals; and

a detector configured to detect reflections of at least some of the pulsed signals, wherein the logic is further executable by the at least one processor to detect the physical object based on the reflections.

17. The computing device of claim 13 , further comprising:

at least one laser configured to emit a three-dimensional lattice; and

at least one detector configured to detect disturbances in the lattice, wherein the logic is further executable by the at least one processor to detect the physical object based on the disturbances.

18. The computing device of claim 13 , wherein the graphical modification of the virtual object further comprises the virtual object changing shape.

19. A non-transitory computer-readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:

displaying a user-interface on a display of a computing device, wherein the user-interface comprises a view region;

displaying a virtual object in the view region, wherein the virtual object is displayed at a focal length from the display along a first line of sight;

determining a physical distance, along the first line of sight, between the display and a physical object;

determining that the focal length is substantially equal to the physical distance to the physical object, such that the virtual object and the physical object appear substantially co-located in the view region; and

responsive to the determination that the focal length is substantially equal to the physical distance, initiating a collision action between the virtual object and the physical object, wherein the collision action comprises a graphical modification of the virtual object, and wherein the graphical modification of the virtual object comprises the computing device causing the virtual object to move away from the physical object.

20. The non-transitory computer-readable medium of claim 19 , wherein the user-interface further comprises a content region that is located outside of the view region, and wherein the user-interface is initially configured such that the at least one content region is arranged along an at least partial ring above the view region.

21. The method of claim 1 , wherein the focal length is a perceived distance from the viewing location to the virtual object.

Assignments (2)
CHANGE OF NAME Recorded Feb 21, 2018
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 045404/0942 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 26, 2017
From: WONG, ADRIAN; MIAO, XIAOYU
To: GOOGLE INC.
Reel/Frame 043706/0394 →
Continuity (3)
Continuation 14626294 · Feb 19, 2015
Continuation 13253695 · Oct 5, 2011
Related Publication 20180024363A1 · Jan 25, 2018