IP Library › Granted Patent US 11,151,791
Granted Patent B2
US 11,151,791 · App. 16/386,398 · Granted Oct 19, 2021

R-snap for production of augmented realities

Inventors: Roger Ray Skidmore (Austin, TX); Blair Nelson Ahlquist (Eugene, OR); Dragomir Rosson (Webster, TX)
Assignee: EDX Technologies, Inc.
G06T19/006G06T15/20G06T17/00G06T2200/04G06T2200/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,151,791
App. No.
16/386,398
Granted
Oct 19, 2021
Kind
B2
Abstract

Images and/or videos have associated therewith information like location and orientation information for the camera used to captured the images/videos. The associated location and orientation (or pose) information facilitates subsequent processing for producing accurate and convincing augmented reality (AR) outputs. In addition, some embodiments associate user-specific information with images or videos for producing customized AR content on a user-to-user basis.

Claims (47)

1. A method for augmented reality (AR), comprising

receiving, by one or more processors, an image or video containing real world content;

embedding, by the one or more processors, perspective information with the image or video as metadata that moves, copies, and transfers with the image or video, wherein the perspective information describes a camera at a first time the camera captured the image or video;

associating, by the one or more processors, user-specific metadata with the image or video on a per user basis, wherein the user-specific metadata moves, copies, and transfers with the image or video;

moving, copying, or transferring the image or video from a first storage medium to a second storage medium different from the first storage medium;

producing at a second time, by the one or more processors, user-specific AR content comprising virtual augmentations overlaid onto the image or video based on the perspective information and the user-specific metadata, wherein the second time is after the first time,

wherein the perspective information is different from the user-specific metadata.

2. The method of claim 1 , wherein the perspective information embedded with the image or video comprises one or more of location, orientation, position, and field of view for the camera at the first time when the camera captured the image or video.

3. The method of claim 2 , wherein the perspective information embedded with the image or video comprises location and orientation information.

4. The method of claim 1 , wherein the user-specific metadata associated with the image or video differentiates between at least three user types including i) entities responsible for originally capturing the image or video, ii) entities responsible for creating virtual content included in the AR content of the producing step, and iii) entities intended to consume the user-specific AR content produced in the producing step.

5. The method of claim 1 , wherein the producing step produces different AR content based on user-specific criteria including one or more a classification, a type, an age, an access level, a demographic, a status, a customer status, and a profession.

6. A method for augmented reality (AR), comprising

collecting or receiving real world perspective information used or usable to define a viewing frustum for an image or video containing real world content;

embedding the perspective information with the image or video as metadata that moves, copies, and transfers with the image or video;

moving, copying, or transferring the image or video from a first storage medium to a second storage medium different from the first storage medium;

applying the viewing frustum to a virtual model configured to model the real world; and

selecting augmentations to include in an AR output, the selection comprising

including for selection only augmentations corresponding with objects that are within a space within the virtual model defined by the applied viewing frustum, and

including for selection only augmentations meeting pre-determined user-specific criteria based on user-specific metadata that moves, copies, and transfers with the image or video,

wherein the AR output is the selected augmentations overlaid onto the image or video,

wherein the perspective information is different from the user-specific metadata.

7. The method of claim 1 , wherein the producing step comprises sourcing user-specific AR content from a virtual model that includes virtual representations of at least some of the real world content of the image or video.

8. The method of claim 1 , wherein the producing step comprises

determining a real world frustum based on the perspective information,

applying the real world frustum to a virtual world to identify virtual object candidates, the virtual world being modeled after the real world and including at least one virtual location corresponding with at least one real world location that appears in the real world content of the image or video; and

selecting augmentations to apply in the producing step based on the virtual object candidates.

9. The method of claim 8 , wherein the virtual object candidates are those objects of the virtual world which are within a space the boundaries of which are defined by the applied frustum.

10. The method of claim 6 , further comprising a step of outputting the AR output to one or more users who correspond with the pre-determined user-specific criteria.

11. The method of claim 6 , wherein the user-specific criteria differentiate between at least three user types including i) entities responsible for originally capturing the image or video, ii) entities responsible for creating virtual content included in the AR content of the producing step, and iii) entities intended to consume the user-specific AR content produced in the producing step.

12. The method of claim 6 , wherein the user-specific criteria include one or more a classification, a type, an age, an access level, a demographic, a status, a customer status, and a profession.

13. A system of AR engines or AR-enabled devices comprising processors configured to execute computer-readable instructions which, when executed, cause the AR engines or AR-enabled devices to

receive, by one or more of the processors, an image or video containing real world content;

embed, by the one or more processors, perspective information with the image or video as metadata that moves, copies, and transfers with the image or video, wherein the perspective information describes a camera at a first time the camera captured the image or video;

associate, by the one or more processors, user-specific metadata with the image or video on a per user basis, wherein the user-specific metadata moves, copies, and transfers with the image or video;

move, copy, or transfer the image or video from a first storage medium to a second storage medium different from the first storage medium;

produce at a second time, by the one or more processors, user-specific AR content comprising virtual augmentations overlaid onto the image or video based on the perspective information and the user-specific metadata, wherein the second time is after the first time,

wherein the perspective information is different from the user-specific metadata.

14. The system of claim 13 , wherein the perspective information embedded with the image or video comprises one or more of location, orientation, position, and field of view for the camera at the time when the camera captured the image or video.

15. The system of claim 14 , wherein the perspective information embedded with the image or video comprises location and orientation information.

16. The system of claim 13 , wherein the user-specific metadata associated with the image or video differentiates between at least three user types including i) entities responsible for originally capturing the image or video, ii) entities responsible for creating virtual content included in the AR content of the producing step, and iii) entities intended to consume the user-specific AR content produced in the producing step.

17. The system of claim 13 , wherein the producing step produces different AR content based on user-specific criteria including one or more a classification, a type, an age, an access level, a demographic, a status, a customer status, and a profession.

18. The system of claim 13 , wherein the producing step comprises sourcing user-specific AR content from a virtual model that includes virtual representations of at least some of the real world content of the image or video.

19. The system of claim 13 , wherein the producing step comprises

determining a real world frustum based on the perspective information,

applying the real world frustum to a virtual world to identify virtual object candidates, the virtual world being modeled after the real world and including locations corresponding with at least one real world location that appears in the real world content of the image or video, and

selecting augmentations to apply in the producing step based on the virtual object candidates.

20. The system of claim 19 , wherein the virtual object candidates are those objects of the virtual world which are within a space the boundaries of which are defined by the applied frustum.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2019
From: SKIDMORE, ROGER RAY; AHLQUIST, BLAIR NELSON; ROSSON, DRAGOMIR
To: EDX TECHNOLOGIES, INC.
Reel/Frame 049208/0323 →
Continuity (2)
Provisional Application 62658871 · Apr 17, 2018
Related Publication 20190318543A1 · Oct 17, 2019
Cited By (1)
US 12,515,137