IP Library Granted Patent US 9,480,919
Granted Patent B2
US 9,480,919 · App. 12/257,832 · Granted Nov 1, 2016

Reconfiguring reality using a reality overlay device

Inventors: Chris Kalaboukis (Los Gatos, CA); Stephan Douris (San Jose, CA); Marc Perry (San Francisco, CA); Barry Crane (Menlo Park, CA)
Assignee: Excalibur IP, LLC
A63F13/53A63F13/02A63F13/12A63F13/65A63F13/90G06F3/011G06F3/04815G06T19/006A63F13/213A63F2300/105A63F2300/1043A63F2300/1093A63F2300/205A63F2300/303A63F2300/69A63F2300/8082G09G3/001G09G5/003
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,480,919
App. No.
12/257,832
Granted
Nov 1, 2016
Kind
B2
Abstract

Virtual entities are displayed alongside real world entities in a wearable reality overlay device worn by the user. Information related to an environment proximate to the wearable device is determined. For example, a position of the wearable device may be determined, a camera may capture an image of the environment, etc. Virtual entity image information representative of an entity desired to be virtually displayed is processed based on the determined information. An image of the entity is generated based on the processed image information as a non-transparent region of a lens of the wearable device, enabling the entity to appear to be present in the environment to the user. The image of the entity may conceal a real world entity that would otherwise be visible to the user through the wearable device. Other real world entities may be visible to the user through the wearable device.

Claims (80)

1. A method comprising:

determining, by at least one processing unit, information related to a wearable device and one or more real-world entities located in an environment proximate to and viewable through the wearable device;

receiving, by the at least one processing unit, image information processed based at least on the determined information, the image information being representative of a first entity;

generating, by the at least one processing unit, an image of the first entity based on the processed image information as a first entity display region of a lens of the wearable device to enable the first entity to appear to be present in the environment using the wearable device, the first entity is a virtual first entity;

tracking, by the at least one processing unit, a position of a movable second entity in the environment, the second entity is visible through a transparent region of the lens of the wearable device and is a real-world second entity;

optically aligning, by the at least one processing unit, the image of the virtual first entity on the lens of the wearable device with the movable real-world second entity, the optically aligning comprising performing at least one of positioning and sizing the virtual first entity to conceal at least a portion of the real-world second entity otherwise visible through the lens of the wearable device as the real-world second entity is being tracked.

2. The method of claim 1 , wherein said generating comprises:

projecting the image of the first entity on the lens.

3. The method of claim 1 , wherein said generating comprises:

selectively activating pixels of the lens to form the image of the first entity.

4. The method of claim 1 , wherein said determining information related to a wearable device and one or more real-world entities located in an environment proximate to and viewable through the wearable device comprises:

determining at least one of a location of the wearable device, an orientation of the wearable device, or a speed at which the wearable device is moving.

5. The method of claim 4 , further comprising:

processing the image information representative of the first entity based on the determined at least one of the location of the wearable device, the orientation of the wearable device, or the speed at which the wearable device is moving.

6. The method of claim 1 , wherein said determining information related to a wearable device and one or more real-world entities located in an environment proximate to and viewable through the wearable device comprises:

determining at least one of a location of the second entity, an orientation of the second entity, or a speed at which the second entity is moving.

7. The method of claim 6 , further comprising:

processing the image information representative of the first entity based on the determined at least one of the location of the second entity, the orientation of the second entity, or the speed at which the second entity is moving.

8. The method of claim 1 , wherein said determining information related to a wearable device and one or more real-world entities located in an environment proximate to and viewable through the wearable device comprises:

capturing an image of the second entity; and

processing the captured image to determine a location of the second entity.

9. The method of claim 1 , further comprising:

receiving updated processed image information representative of the first entity that is processed based at least on updated determined information related to the environment proximate to a wearable device; and

generating an updated image of the first entity based on the updated processed image information as an updated first entity display region of the lens.

10. The method of claim 1 , further comprising:

generating a second image of the first entity based on the received image information as a first entity display region of a second lens of the wearable device.

11. The method of claim 1 , wherein the image information is representative of a plurality of entities, wherein said generating comprises:

generating a plurality of images corresponding to the plurality of entities based on the processed image information as a plurality of entity display regions of the lens.

12. The method of claim 1 , wherein the first entity is a playing field, wherein said generating comprises:

generating the image of the playing field to enable the user to view the playing field using the wearable device; and

the method further comprising:

enabling the user to participate in a game configured to take place in the playing field.

13. The method of claim 12 , further comprising:

receiving second image information processed based at least on the determined information, the second image information being representative of an implement of the game; and

generating an image of the implement based on the processed second image information as a second entity display region of the lens of the wearable device to enable the implement of the game to appear to be present in the environment to the user of the wearable device.

14. The method of claim 1 , wherein said tracking comprises:

tracking the position of one or more real-world entities visible through the lens including the real-world second entity; and

wherein said optically aligning comprises:

optically aligning the image of the virtual first entity on the lens with the one or more real-world entities to conceal at least a portion of each of the one or more real-world entities otherwise visible through the lens of the wearable device as the one or more real-world entities are being tracked.

15. A wearable device comprising:

a position monitor configured to determine a position of the wearable device;

a lens; and

a display generator configured to receive image information processed based at least on the determined position and on determined information related to one or more real-world entities located in an environment proximate to and viewable through the wearable device, the image information being representative of a first entity that is a virtual first entity;

wherein the display generator is configured to generate an image of the virtual first entity as a first entity display region of the lens based on the processed image information to enable the virtual first entity to appear to be present in the environment proximate to the wearable device using the wearable device;

wherein the wearable device is configured to track a position of a movable second entity in the environment, the second entity is visible through a transparent region of the lens of the wearable device and is a real-world second entity; and

wherein the display generator is configured to optically align the image of the virtual first entity on the lens of the wearable device with the movable real-world second entity, optically aligning comprising performing at least one of positioning and sizing the virtual first entity to conceal at least a portion of the real-world second entity otherwise visible through the lens of the wearable device as the real-world second entity is being tracked.

16. The wearable device of claim 15 , wherein the display generator comprises:

an image projector configured to project the image of the first entity on the lens.

17. The wearable device of claim 15 , wherein the display generator is a display device that includes an array of image pixels, the lens comprises the display device, the display device is configured to selectively activate pixels of the array of image pixels to form the image of the first entity, and non-activated pixels of the array of image pixels are transparent.

18. The wearable device of claim 15 , wherein the position monitor is further configured to determine at least one of an orientation of the wearable device or a speed of the wearable device.

19. The wearable device of claim 18 , further comprising:

an image processor configured to process the image information representative of the first entity based at least on the determined position of the wearable device, the determined orientation of the wearable device, or the determined speed of the wearable device to generate the processed image information.

20. The wearable device of claim 15 , further comprising:

an image processor configured to process the image information representative of the first entity based at least on a determined position of the second entity, a determined orientation of the second entity, or a speed of the second entity to generate the processed image information.

21. The wearable device of claim 15 , further comprising:

a camera configured to capture an image of the environment;

wherein the image processor is configured to process an image of the second entity captured by the camera to determine a location of the second entity.

22. The wearable device of claim 15 , further comprising:

wherein the display generator is configured to receive updated processed image information representative of the first entity that is processed based at least on updated determined information related to the environment proximate to a wearable device, and to generate an updated image of the first entity based on the updated processed image information as an updated first entity display region of the lens.

23. The wearable device of claim 15 , further comprising:

a second display generator configured to generate a second image of the first entity based on received image information as a first entity display region of a second lens of the wearable device.

24. The wearable device of claim 15 , further comprising:

a game engine;

wherein the first entity is a playing field, wherein the display generator is configured to generate the image of the playing field to enable the user to view the playing field using the wearable device; and

wherein the game engine is configured to enable the user to participate in a game configured to take place in the playing field.

25. The wearable device of claim 24 , wherein the display generator is configured to receive second image information processed based at least on the determined information, the second image information being representative of an implement of the game, and to generate an image of the implement based on the processed second image information as a second entity display region of the lens of the wearable device to enable the implement of the game to appear to be present in the environment to the user of the wearable device.

26. The wearable device of claim 15 , further comprising:

a communication interface configured to transmit information representative of the determined position information from the wearable device, and to receive the image information processed based at least on the determined position.

27. The wearable device of claim 15 , wherein the wearable device is configured to track the position of one or more real-world entities visible through the lens including the real-world second entity; and

wherein the display generator is configured to optically align the image of the virtual first entity on the lens with the one or more real-world entities to conceal at least a portion of each of the one or more real-world entities otherwise visible through the lens of the wearable device as the one or more real-world entities are being tracked.

28. A system comprising:

at least one processor;

a storage medium for tangibly storing thereon program logic for execution by the at least one processor, the stored program logic comprising:

transmitting logic for execution by the at least one processor for transmitting, to a server, information including position information indicative of a location of a wearable device and information regarding one or more real-world entities located in an environment proximate to and viewable through the wearable device;

receiving logic for execution by the at least one processor for receiving, from the server, image information in response to the transmitted information, the received image information representing a first entity that is a virtual first entity;

generating logic for execution by the at least one processor for generating an image of the virtual first entity as a first entity display region of a lens based on the received image information to enable the virtual first entity to appear to be present in the environment proximate to the wearable device using the wearable device;

tracking logic for execution by the at least one processor for tracking a position of a movable second entity in the environment, the second entity is visible through a transparent region of the lens and is a real-world second entity; and

aligning logic for execution by the at least one processor for optically aligning the image of the virtual first entity on the lens of the wearable device with the movable real-world second entity, the optically aligning comprising performing at least one of positioning and sizing the virtual first entity to conceal at least a portion of the real-world second entity otherwise visible through the lens of the wearable device as the real-world second entity is being tracked.

29. The system of claim 28 , the stored program logic further comprising:

enabling logic executed by the at least one processor for enabling a game for a user of the wearable device, the game including the first entity.

Assignments (9)
CORRECTIVE ASSIGNMENT TO CORRECT THE THE ASSIGNOR NAME PREVIOUSLY RECORDED AT REEL: 052853 FRAME: 0153. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Mar 29, 2021
From: R2 SOLUTIONS LLC
To: STARBOARD VALUE INTERMEDIATE FUND LP, AS COLLATERAL AGENT
Reel/Frame 056832/0001 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE NAME PREVIOUSLY RECORDED ON REEL 053654 FRAME 0254. ASSIGNOR(S) HEREBY CONFIRMS THE RELEASE OF SECURITY INTEREST GRANTED PURSUANT TO THE PATENT SECURITY AGREEMENT PREVIOUSLY RECORDED. Recorded Dec 30, 2020
From: STARBOARD VALUE INTERMEDIATE FUND LP
To: R2 SOLUTIONS LLC
Reel/Frame 054981/0377 →
RELEASE OF SECURITY INTEREST IN PATENTS Recorded Jul 8, 2020
From: STARBOARD VALUE INTERMEDIATE FUND LP
To: ACACIA RESEARCH GROUP LLC; AMERICAN VEHICULAR SCIENCES LLC; BONUTTI SKELETAL INNOVATIONS LLC; CELLULAR COMMUNICATIONS EQUIPMENT LLC; INNOVATIVE DISPLAY TECHNOLOGIES LLC; LIFEPORT SCIENCES LLC; LIMESTONE MEMORY SYSTEMS LLC; MOBILE ENHANCEMENT SOLUTIONS LLC; MONARCH NETWORKING SOLUTIONS LLC; NEXUS DISPLAY TECHNOLOGIES LLC; PARTHENON UNIFIED MEMORY ARCHITECTURE LLC; R2 SOLUTIONS LLC; SAINT LAWRENCE COMMUNICATIONS LLC; STINGRAY IP SOLUTIONS LLC; SUPER INTERCONNECT TECHNOLOGIES LLC; TELECONFERENCE SYSTEMS LLC; UNIFICATION TECHNOLOGIES LLC
Reel/Frame 053654/0254 →
PATENT SECURITY AGREEMENT Recorded Jun 5, 2020
From: ACACIA RESEARCH GROUP LLC; AMERICAN VEHICULAR SCIENCES LLC; BONUTTI SKELETAL INNOVATIONS LLC; CELLULAR COMMUNICATIONS EQUIPMENT LLC; INNOVATIVE DISPLAY TECHNOLOGIES LLC; LIFEPORT SCIENCES LLC; LIMESTONE MEMORY SYSTEMS LLC; MERTON ACQUISITION HOLDCO LLC; MOBILE ENHANCEMENT SOLUTIONS LLC; MONARCH NETWORKING SOLUTIONS LLC; NEXUS DISPLAY TECHNOLOGIES LLC; PARTHENON UNIFIED MEMORY ARCHITECTURE LLC; R2 SOLUTIONS LLC; SAINT LAWRENCE COMMUNICATIONS LLC; STINGRAY IP SOLUTIONS LLC; SUPER INTERCONNECT TECHNOLOGIES LLC; TELECONFERENCE SYSTEMS LLC; UNIFICATION TECHNOLOGIES LLC
To: STARBOARD VALUE INTERMEDIATE FUND LP, AS COLLATERAL AGENT
Reel/Frame 052853/0153 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 28, 2018
From: EXCALIBUR IP, LLC.
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 045065/0823 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 3, 2016
From: YAHOO! INC.
To: EXCALIBUR IP, LLC
Reel/Frame 038950/0592 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 1, 2016
From: EXCALIBUR IP, LLC
To: YAHOO! INC.
Reel/Frame 038951/0295 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 18, 2016
From: YAHOO! INC.
To: EXCALIBUR IP, LLC
Reel/Frame 038383/0466 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 24, 2008
From: KALABOUKIS, CHRIS; DOURIS, STEPHAN; PERRY, MARC; CRANE, BARRY
To: YAHOO! INC.
Reel/Frame 021734/0874 →
Continuity (1)
Related Publication 20100103075A1 · Apr 29, 2010