IP Library Granted Patent US 9,001,252
Granted Patent B2
US 9,001,252 · App. 12/611,083 · Granted Apr 7, 2015

Image matching to augment reality

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,001,252
App. No.
12/611,083
Granted
Apr 7, 2015
Kind
B2
Abstract

Technology for augmenting reality using image matching is described. The technology can receive a captured image, determine location information relating to the received image, and transmit the location information to a server computing device. The technology may then receive one or more tags from the server computing device, wherein each tag can include one or more patches and supplementary information corresponding to at least one of the patches and each patch can be associated with the location information. The technology can then determine using an image matching process whether at least one of the received one or more patches appears in the received image, and if a patch appears in the received image, render the supplementary information corresponding to the patch.

Claims (40)

1. A method performed by a computing system for augmenting reality, comprising:

receiving a captured image;

determining location information relating to the received captured image;

transmitting the determined location information to a server computing device;

receiving one or more tags from the server computing device, each tag including one or more patches and supplementary information corresponding to at least one of the patches, each patch including a patch image and location information that specifies a location that is near a location specified by the determined location information;

determining, using a template matching process, whether at least one of the received one or more patch images appears in the received captured image; and

in response to a determination that a first patch image appears in the received image, rendering the supplementary information corresponding to the patch associated with the first patch image.

2. The method of claim 1 , wherein the captured image is received from a digital camera associated with the computing system.

3. The method of claim 1 , wherein the determining the location information is based on information received from a GPS signal.

4. The method of claim 1 , wherein the location information transforms a physical condition sensed by an accelerometer into an electrical signal.

5. The method of claim 1 , wherein the supplementary information is associated with additional information that specifies how to render the supplementary information.

6. The method of claim 1 , wherein the template matching process is at least partially performed by a hardware-based video compression processor.

7. The method of claim 1 , wherein the rendering includes superimposing an image specified by the supplementary information.

8. The method of claim 1 , wherein the rendering includes playing an audible sound specified by the supplementary information.

9. The method of claim 1 , further comprising:

identifying a confidence level for the patch image, and

discarding the patch image in response to the patch image having an identified confidence level below a specified threshold confidence level.

10. A computer-readable memory storing computer-executable instructions that, when executed, perform a method for augmenting reality, the method comprising:

receiving a captured digital image that is digitized by an image capturing device;

determining location information for the image capturing device at the time the captured digital image is digitized;

transmitting the determined location information;

receiving one or more tags, each tag including one or more patches, each patch including a patch image and location information that specifies a location that is near a location specified by the determined location information;

determining using a template matching process whether at least one of the tags includes a patch image that appears in the captured digital image; and

in response to a determination that a first patch image appears in the captured digital image, rendering supplementary information corresponding to the patch associated with the first patch image.

11. The computer-readable memory of claim 10 , further comprising storing the captured digital image, the first patch image, and the supplementary information in a data structure, wherein the supplementary information specifies information for the rendering in response to the determination that the first patch image appears in the captured digital image.

12. The computer-readable memory of claim 11 , wherein the supplementary information includes an image to superimpose on the digital image.

13. The computer-readable memory of claim 11 , wherein the supplementary information includes an audible sound.

14. The computer-readable memory of claim 10 , wherein at least one of the one or more tags comprises the supplementary information.

15. The computer-readable memory of claim 14 , wherein the supplementary information includes additional information specifying rendering information.

16. The computer-readable memory of claim 10 , wherein the determined location information is transmitted using a wireless data communications protocol supported by a mobile telephone service.

17. A system for augmenting reality, comprising:

a processor and memory;

an image capturing component;

a hardware component configured to detect position information in response to an image being captured by the image capturing component;

a transceiver configured to transmit the detected position information and to receive at least one tag, wherein a tag comprises a template image, location information that specifies a location that is near a location specified by the detected location information, and associated information; and

an imaging component configured to employ a template matching routine to determine whether the template image appears within the captured image and to render supplementary information responsive to a determination that the template image appears within the captured image.

18. The system of claim 17 , wherein the image capturing component includes a digital camera associated with a client computing device with which the processor and memory are associated.

19. The system of claim 17 , wherein the hardware component includes at least one of a GPS receiver, an accelerometer, or a compass.

20. The system of claim 17 , wherein the imaging component is further configured to function in association with a second hardware component configured to implement video compression specification.

21. The system of claim 17 , wherein the supplementary information is rendered by superimposition of an image on the captured image.

Assignments (7)
RELEASE OF SECURITY INTEREST IN PATENTS, RECORDED ON JANUARY 29, 2019 AT REEL 048373 FRAME 0217 Recorded Sep 22, 2025
From: CRESTLINE DIRECT FINANCE, L.P., AS COLLATERAL AGENT
To: EMPIRE TECHNOLOGY DEVELOPMENT LLC
Reel/Frame 072936/0464 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 18, 2020
From: EMPIRE TECHNOLOGY DEVELOPMENT, LLC
To: HUAWEI DEVICE CO., LTD.
Reel/Frame 052144/0211 →
RELEASE OF SECURITY INTEREST Recorded Mar 13, 2020
From: CRESTLINE DIRECT FINANCE, L.P.
To: EMPIRE TECHNOLOGY DEVELOPMENT LLC
Reel/Frame 052112/0425 →
SECURITY INTEREST Recorded Jan 29, 2019
From: EMPIRE TECHNOLOGY DEVELOPMENT LLC
To: CRESTLINE DIRECT FINANCE, L.P.
Reel/Frame 048373/0217 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 23, 2012
From: INTELLECTUAL VENTURES ASIA PTE. LTD.
To: EMPIRE TECHNOLOGY DEVELOPMENT LLC
Reel/Frame 027916/0457 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 23, 2012
From: GLITTER TECHNOLOGY LLP
To: INTELLECTUAL VENTURES ASIA PTE. LTD.
Reel/Frame 027916/0231 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 2, 2009
From: HANNAFORD, BLAKE
To: EMPIRE TECHNOLOGY DEVELOPMENT LLC
Reel/Frame 023459/0001 →