IP Library Granted Patent US 10,664,518
Granted Patent B2
US 10,664,518 · App. 16/168,419 · Granted May 26, 2020

Wide area augmented reality location-based services

Inventors: David McKinnon (San Francisco, CA); Kamil Wnuk (Playa del Rey, CA); Jeremi Sudol (New York, NY); Matheen Siddiqui (Culver City, CA); John Wiacek (Los Angeles, CA); Bing Song (La Canada, CA); Nicholas J. Witchey (Laguna Hills, CA)
Assignee: Nant Holdings IP, LLC
G06F16/5866G06F16/29G06F16/50G06F16/58G06F16/5854G06F16/9535G06T15/20G06T19/003G06T19/006H05K999/99G06T2219/024
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,664,518
App. No.
16/168,419
Granted
May 26, 2020
Kind
B2
Abstract

Apparatus, methods and systems of providing AR content are disclosed. Embodiments of the inventive subject matter can obtain an initial map of an area, derive views of interest, obtain AR content objects associated with the views of interest, establish experience clusters and generate a tile map tessellated based on the experience clusters. A user device could be configured to obtain and instantiate at least some of the AR content objects based on at least one of a location and a recognition.

Claims (61)

1. A device capable of rendering augmented reality (AR), the device comprising:

at least one sensor, including a location sensor;

a display;

a non-transitory computer readable memory storing software instructions; and

at least one processor coupled with the non-transitory computer readable memory, the at least one sensor, and the display; and, upon execution of the software instructions, is configurable to:

obtain sensor data from the at least one sensor wherein the sensor data includes a device location obtained from the location sensor;

obtain an area of interest via an area database based on at least the device location within the sensor data;

access an area tile map of the area of interest, the area tile map represented by a set of tile subareas that includes one or more tessellated tiles from a tessellated tile map;

identify a tile subarea from the set of tile subareas based at least in part on the device location relative to one or more locations of tile subareas from the set of tile subareas, wherein the identified tile subarea covers at least a portion of the area of interest, and wherein one or more tessellated tiles within the identified tile subarea are associated with one or more AR content objects;

populate the non-transitory computer readable memory with at least one of the one or more AR content objects associated with the one or more tessellated tiles corresponding with the identified tile subarea; and

render the at least one of the one or more AR content objects that is associated with the identified tile subarea on the display based on a view of interest.

2. The device of claim 1 , wherein the location sensor comprises at least one of the following location sensors: a global positioning system (GPS) sensor, an image sensor, and a wireless sensor.

3. The device of claim 1 , wherein the device location includes at least one of the following: a global positioning system (GPS) location, a wireless signal location, a depth of field location, and an image-based simultaneous location and mapping (SLAM) location.

4. The device of claim 1 , wherein the at least one sensor further includes one or more of the following: a camera, an accelerometer, and a gyroscope.

5. The device of claim 1 , wherein the sensor data further includes at least one of device position data and device orientation data.

6. The device of claim 5 , wherein the at least one of the one or more AR content objects is rendered based on the view of interest, wherein the view of interest is derived from at least one of the device position data and the device orientation data.

7. The device of claim 1 , wherein the at least one of the one or more AR content objects is rendered on the display as an overlay of an image related to the real-world.

8. The device of claim 1 , wherein the area of interest comprises one of the following: a landmark, user-defined boundaries, state-defined boundaries, natural boundaries, a city, a country, a business, a shopping center, a warehouse, a stadium, a wilderness area, a road, a garden, a zoo, an amusement park, a beach and a building.

9. The device of claim 1 , wherein the area of interest comprises a restaurant.

10. The device of claim 1 , wherein the area of interest comprises a coffee shop.

11. The device of claim 1 , wherein the area of interest comprises an outdoor setting.

12. The device of claim 1 , wherein the device communicatively couples with the area database over a network.

13. The device of claim 1 , wherein the at least one of the one or more AR content objects is obtained from an AR content database.

14. The device of claim 13 , wherein the device is communicatively coupled with the AR content database over a network.

15. The device of claim 1 , wherein the at least one of the one or more AR content objects is rendered relative to a recognized object.

16. The device of claim 15 , wherein the recognized object represents an attachment point for the at least one of the one or more AR content objects.

17. The device of claim 16 , wherein the attachment point includes at least one of the following: a billboard, a wall, a floor, a plant, a sign, a logo, a landmark, a building, and a car.

18. The device of claim 1 , wherein the at least one of the one or more AR content objects comprises a game object.

19. The device of claim 18 , wherein the at least one of the one or more AR content objects comprises an interactive game experience.

20. The device of claim 1 , wherein the at least one of the one or more AR content objects comprises at least one of the following: a virtual object, a digital image, a digital video, audio data, an application, a script, a promotion, an advertisement, a graphic, a sprite, and an animation.

21. The device of claim 1 , wherein the at least one of the one or more AR content objects comprises an exclusive content related to the identified tile subarea.

22. The device of claim 21 , wherein the exclusive content related to the identified tile subarea is rendered based on a paid fee.

23. The device of claim 1 , wherein the non-transitory computer readable memory, the at least one processor, the at least one sensor, and the display comprise at least one of the following: a smart phone, augmented reality goggles, a watch, a tablet, a drone, a vehicle, a wearable augmented reality device, and a robot.

24. The device of claim 1 , wherein the at least one of the one or more AR content objects is determined based on the device's movement relative to the area of interest or the identified tile subarea.

25. The device of claim 24 , wherein the at least one of the one or more AR content objects is auto-populated in the non-transitory computer readable memory and rendered based on the movement relative to the area of interest or the identified tile subarea.

26. The device of claim 1 , wherein the identified tile subarea or the at least one of the one or more AR content objects is identified based on time.

27. The device of claim 1 , wherein a modification of the at least one of the one or more AR content objects by a first user is viewable by a subset of users.

28. The device of claim 1 , wherein a virtual item is made available to a user when the user executes a navigation event with respect to the area of interest or the identified tile subarea.

29. The device of claim 28 , wherein the navigation event comprises at least one of the following: the user being located in the area of interest at any time, the user being located in the area of interest for a predetermined minimum amount of time, the user being located in the area of interest at a predetermined time, or the user being located in the area of interest and capturing an image viewable from the area of interest.

30. The device of claim 1 , wherein a virtual item is provided to a first user and a second user when the first user and the second user execute a navigation event with respect to the area of interest or the identified tile subarea.

31. The device of claim 30 , wherein the navigation event comprises the first user and the second user each performing at least one of the following: being located in the area of interest at any time, being located in the area of interest for a predetermined minimum amount of time, being located in the area of interest at a predetermined time, or being located in the area of interest and capturing an image viewable from the area of interest.

32. The device of claim 1 , wherein the at least one processor, upon execution of the software instructions, is further configurable to obtain the area tile map of the area of interest.

33. The device of claim 1 , wherein the at least one processor, upon execution of the software instructions, is further configurable to generate the area tile map of the area of interest.

34. A method of presenting augmented reality (AR) with a computing device, the method comprising:

obtaining sensor data from at least one sensor, the at least one sensor including a location sensor, wherein the sensor data includes a device location of the computing device obtained from the location sensor;

obtaining an area of interest via an area database based on at least the device location within the sensor data;

accessing an area tile map of the area of interest, the area tile map represented by a set of tile subareas that includes one or more tessellated tiles from a tessellated tile map;

identifying a tile subarea from the set of tile subareas based at least in part on the device location relative to one or more locations of tile subareas from the set of tile subareas, wherein the identified tile subarea covers at least a portion of the area of interest, and wherein one or more tessellated tiles within the identified tile subarea are associated with one or more AR content objects;

populating a non-transitory computer readable memory of the computing device with at least one of the one or more AR content objects associated with the one or more tessellated tiles corresponding with the identified tile subarea; and

rendering the at least one of the one or more AR content objects that is associated with the identified tile subarea on a display of the computing device based on a view of interest.

35. The method of claim 34 , wherein the location sensor comprises at least one of the following location sensors: a global positioning system (GPS) sensor, an image sensor, and a wireless sensor.

36. The method of claim 34 , wherein the device location includes at least one of the following: a global positioning system (GPS) location, a wireless signal location, a depth of field location, and an image-based simultaneous location and mapping (SLAM) location.

37. A computer program product embedded in at least one non-transitory computer-readable medium comprising instructions for rendering augmented reality (AR) content on a computing device, which, when executed, configure one or more computer processors to perform a method comprising:

obtaining sensor data from at least one sensor, the at least one sensor including a location sensor, wherein the sensor data includes a device location of the computing device obtained from the location sensor;

obtaining an area of interest via an area database based on at least the device location within the sensor data;

accessing an area tile map of the area of interest, the area tile map represented by a set of tile subareas that includes one or more tessellated tiles from a tessellated tile map;

identifying a tile subarea from the set of tile subareas based at least in part on the device location relative to one or more locations of tile subareas from the set of tile subareas, wherein the identified tile subarea covers at least a portion of the area of interest, and wherein one or more tessellated tiles within the identified tile subarea are associated with one or more AR content objects;

populating a non-transitory computer readable memory of the computing device with at least one of the one or more AR content objects associated with the one or more tessellated tiles corresponding with the identified tile subarea; and

rendering the at least one of the one or more AR content objects that is associated with the identified tile subarea on a display of the computing device based on a view of interest.

38. The computer program product of claim 37 , wherein the location sensor comprises at least one of the following location sensors: a global positioning system (GPS) sensor, an image sensor, and a wireless sensor.

39. The computer program product of claim 37 , wherein the device location includes at least one of the following: a global positioning system (GPS) location, a wireless signal location, a depth of field location, and an image-based simultaneous location and mapping (SLAM) location.

Assignments (10)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2018
From: MCKINNON, DAVID
To: NANT VISION, INC.
Reel/Frame 047284/0889 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2018
From: WNUK, KAMIL
To: NANT VISION, INC.
Reel/Frame 047285/0024 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2018
From: SUDOL, JEREMI
To: NANT VISION, INC.
Reel/Frame 047285/0082 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2018
From: SIDDIQUI, MATHEEN
To: NANT VISION, INC.
Reel/Frame 047285/0100 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2018
From: WIACEK, JOHN
To: NANTMOBILE, LLC
Reel/Frame 047285/0116 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2018
From: SONG, BING
To: NANTWORKS, LLC
Reel/Frame 047285/0127 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2018
From: WITCHEY, NICHOLAS J
To: NANTWORKS, LLC
Reel/Frame 047285/0143 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2018
From: NANT VISION, INC.
To: NANT HOLDINGS IP, LLC
Reel/Frame 047285/0204 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2018
From: NANTMOBILE, LLC
To: NANT HOLDINGS IP, LLC
Reel/Frame 047285/0207 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2018
From: NANTWORKS, LLC
To: NANT HOLDINGS IP, LLC
Reel/Frame 047285/0229 →
Continuity (5)
Continuation 15794993 · Oct 26, 2017
Continuation 15406146 · Jan 13, 2017
Continuation 14517728 · Oct 17, 2014
Provisional Application 61892238 · Oct 17, 2013
Related Publication 20190057113A1 · Feb 21, 2019
Cited By (5)
US 12,201,048 US 12,369,513 US 12,406,441 US 12,439,840 US 12,694,631