IP Library Granted Patent US 9,639,964
Granted Patent B2
US 9,639,964 · App. 13/840,742 · Granted May 2, 2017

Dynamically preserving scene elements in augmented reality systems

Inventors: Gene Fein (Westlake, CA); Royce A. Levien (Lexington, MA); Richard T. Lord (Tacoma, WA); Robert W. Lord (Seattle, WA); Mark A. Malamud (Seattle, WA); John D. Rinaldo, Jr. (Bellevue, WA); Clarence T. Tegreene (Mercer Island, WA)
Assignee: Elwha LLC
G06T11/00G06F3/011
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,639,964
App. No.
13/840,742
Granted
May 2, 2017
Kind
B2
Abstract

Methods, apparatuses, computer program products, devices and systems are described that carry out accepting a user request associated with at least one of an item, an aspect, or an element of a field of view of an augmented reality device; determining that a first presentation of the at least one item, aspect, or element has a limited period of viability for user interaction relative to the field of view of the augmented reality device; and at least one of maintaining the first presentation or providing a substantially similar second presentation in response to determining that a first presentation of the at least one item, aspect, or element has a limited period of viability for interaction relative to the field of view of the augmented reality device.

Claims (85)

1. A system comprising:

circuitry configured for receiving, via an augmented reality device, a user input indicating a request to interact with a real-world object within a field of view of the augmented reality device;

circuitry configured for determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device by:

detecting movement of the real-world object relative to the field of view of the augmented reality device;

determining a time interval that the real-world object will remain within the field of view of the augmented reality device; and

comparing the time interval that the real-world object will remain within the field of view of the augmented reality device to a threshold time interval; and

circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device.

2. The system of claim 1 , wherein the circuitry configured for receiving, via an augmented reality device, a user input indicating a request to interact with a real-world object within a field of view of the augmented reality device comprises:

circuitry configured for accepting at least one of touch input, finger pointing input, voice input, hand gesture input, finger gesture input, combined touch input of a touch sensitive portion and pointing input, or eye movement input.

3. The system of claim 1 , wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for preserving a displayed indicia of at least one of a person, an object, an animal, a sign, a building, a vehicle, a display, a perspective, or a condition on the augmented reality device.

4. The system of claim 1 , wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device based at least partly on at least one of a user task, a system task, a context, a known user interest, or a known user preference.

5. The system of claim 1 , wherein the circuitry configured for determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for determining that a moving object has a limited period of visibility within the field of view of the augmented reality device.

6. The system of claim 1 , wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for freezing a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device.

7. The system of claim 1 , wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for preserving at least one displayed simulated representation of the real-world object.

8. The system of claim 1 , further comprising:

circuitry configured for performing image recognition to identify the real-world object based on at least of: a context of the augmented reality device, a device in proximity to the augmented reality device, an operation running on the augmented reality device, a task associated with the augmented reality device, a known user interest, or a known user preference.

9. The system of claim 1 , further comprising:

circuitry configured for performing image recognition to identify the real-world object; and

circuitry configured for displaying at least one menu item or command available for the real-world object responsive to recognition of the real world object.

10. The system of claim 1 , wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for highlighting a displayed view of the real-world object.

11. The system of claim 1 , wherein the circuitry configured for receiving, via an augmented reality device, a user input indicating a request to interact with a real-world object within a field of view of the augmented reality device comprises:

circuitry configured for detecting a user action at a location within a field of view of the augmented reality device corresponding to the real-world object.

12. The system of claim 1 , further comprising:

circuitry configured for dragging the indicia of the real-world object within a displayed view of the field of view of the augmented reality device responsive to a user action.

13. The system of claim 1 , wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for preserving at least one of a displayed image, cartoon, frozen scene portion, or simulation of the real-world object on the augmented reality device.

14. The system of claim 1 , wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for preserving a displayed indicia of the real-world object on at least one of a smartphone, tablet, camera, augmented reality eyeglasses, or smartwatch.

15. The system of claim 1 , wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for preserving a displayed indicia of the real-world object at a location within a displayed view of the field of view of the augmented reality device corresponding to at least one of a touch input, finger pointing input, voice input, hand gesture input, finger gesture input, or eye movement input.

16. The system of claim 1 , wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for preserving at least one of a photo, symbol, pixel, or drop destination corresponding to the real-world object.

17. The system of claim 1 , further comprising:

circuitry configured for determining the displayed indicia of the real-world object based at least partly on a geographic location of the augmented reality device.

18. The system of claim 1 , further comprising:

circuitry configured for performing at least one of the following specified operations associated with the displayed indicia of the real-world object: playing media, implementing a check-out, queuing media, posting to a social media site, completing a transaction, collecting an item, presenting content for viewing, or enabling game play.

19. The system of claim 1 , further comprising:

circuitry configured for transferring data associated with the real-world object to a computing device in the field of view of the augmented reality device at least partly in response to movement of the displayed indicia of the real-world object to a position that corresponds to the computing device in the field of view of the augmented reality device.

20. The system of claim 1 , wherein the determining a time interval that the real-world object will remain within the field of view of the augmented reality device comprises:

determining at least one of a speed, acceleration, or trajectory of the real-world object relative to the field of view of the augmented reality device.

21. The system of claim 1 , wherein the determining a time interval that the real-world object will remain within the field of view of the augmented reality device comprises:

determining a time interval until the real-world object will move behind an obstacle.

22. A computer-implemented method comprising:

receiving, via an augmented reality device, a user input indicating a request to interact with a real-world object within a field of view of the augmented reality device;

determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device by:

detecting movement of the real-world object relative to the field of view of the augmented reality device;

determining a time interval that the real-world object will remain within the field of view of the augmented reality device; and

comparing the time interval that the real-world object will remain within the field of view of the augmented reality device to a threshold time interval; and

preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device.

23. A system comprising:

an augmented reality device including at least:

at least one sensor operable to at least detect one or more user inputs;

a display operable to display an augmented reality scene that includes one or more real-world objects and one or more augmented reality representations; and

one or more wireless communication units operable to transmit or receive one or more wireless communications; and

one or more instructions on at least one media that when executed on a computing device associated with the augmented reality device program the computing device for:

receiving, via the augmented reality device, a user input indicating a request to interact with a real-world object within a field of view of the augmented reality device;

determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device by:

detecting movement of the real-world object relative to the field of view of the augmented reality device;

determining a time interval that the real-world object will remain within the field of view of the augmented reality device; and

comparing the time interval that the real-world object will remain within the field of view of the augmented reality device to a threshold time interval; and

preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device.

24. A system comprising:

means for receiving, via an augmented reality device, a user input indicating a request to interact with a real-world object within a field of view of the augmented reality device;

means for determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device by:

detecting movement of the real-world object relative to the field of view of the augmented reality device;

determining a time interval that the real-world object will remain within the field of view of the augmented reality device; and

comparing the time interval that the real-world object will remain within the field of view of the augmented reality device to a threshold time interval; and

means for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device.

25. The system of claim 1 , wherein the circuitry configured for receiving, via an augmented reality device, a user input indicating a request to interact with a real-world object within a field of view of the augmented reality device comprises:

circuitry configured for detecting, with a camera of the augmented reality device, at least one of a hand gesture, a finger gesture or an eye movement corresponding to the real-world object.

26. The system of claim 1 , further comprising:

circuitry configured for releasing the displayed indicia of the real-world object such that displayed indicia of the real-world object transitions through animation to a live view.

27. A system comprising:

circuitry configured for receiving, via an augmented reality device, a user input indicating a request to interact with a real-world object within a field of view of the augmented reality device;

circuitry configured for determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device by:

detecting movement of the real-world object relative to the field of view of the augmented reality device;

determining a time interval that the real-world object will have at least a threshold viewable size within the field of view of the augmented reality device; and

comparing the time interval that the real-world object will have at least the threshold viewable size within the field of view of the augmented reality device to a threshold time interval; and

circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 3, 2013
From: FEIN, GENE; LEVIEN, ROYCE A.; LORD, RICHARD T.; LORD, ROBERT W.; MALAMUD, MARK A.; RINALDO, JOHN D., JR.; TEGREENE, CLARENCE T.
To: ELWHA LLC
Reel/Frame 030738/0827 →
Continuity (1)
Related Publication 20140267409A1 · Sep 18, 2014