IP Library › Granted Patent US 10,380,394
Granted Patent B1
US 10,380,394 · App. 15/893,165 · Granted Aug 13, 2019

Presenting an augmented reality within a custom graphic

Inventors: Nathan Kenneth Boyd (Los Angeles, CA); Christopher Jonathan Luc (Venice, CA); Kirk Ouimet (Orem, UT); Rahul Bhupendra Sheth (Los Angeles, CA)
Assignee: Snap Inc.
G06K7/1456G06F3/04842G06F16/5866G06K7/1417G06K7/1443G06K7/1447G06K19/06037G06K19/06103G06K19/06112G06T11/60H04L51/04H04L67/306H04L67/42
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,380,394
App. No.
15/893,165
Filed
Feb 9, 2018
Granted
Aug 13, 2019
Kind
B1
Art Unit
2876
USPC
235/462.11
Abstract

Systems and methods for presenting an augmented reality within a custom graphic are provided. In example embodiments, one or more hardware processors may receive an image, identify candidate shape features in the image, identify a custom graphic by comparing the candidate shape features with a reference shape feature of the custom graphic, identify an optical barcode based on the identified custom graphic, identify an augmented reality based on the optical barcode, and transmit the augmented reality to the client device.

Claims (71)

1. A method comprising:

receiving, by one or more hardware processors, an image from a client device;

identifying, by the one or more hardware processors, candidate shape features from the image;

identifying, by the one or more hardware processors, a custom graphic by comparing the candidate shape features with a reference shape feature of the custom graphic, the comparing comprising:

computing an area score by comparing a first candidate shape feature of the candidate shape features that represents an area value with a reference area of the reference shape;

determining that the area score fails to exceed an area rule threshold; and

analyzing a second candidate shape feature of the candidate shape features to identify the custom graphic in response to determining that the area score fails to exceed the area rule threshold; and

identifying, by the one or more hardware processors, an optical barcode based on the identified custom graphic;

identifying, by the one or more hardware processors, an augmented reality based on the optical barcode; and

transmitting, by the one or more hardware processors, the augmented reality to the client device

identifying, by the one or more hardware processors, a user account based on the optical barcode;

receiving, by the one or more hardware processors, an indication that the image including the optical barcode was selected at the client device; and

transmitting, to the client device, a user interface item associated with the user account in response to the indication.

2. The method of claim 1 , further comprising determining an orientation of the custom graphic based on the image, wherein the identification of the optical barcode is based on the orientation.

3. The method of claim 1 , further comprising generating the augmented reality to include a video, the video configured to be displayed within the custom graphic.

4. The method of claim 1 , further comprising:

generating the augmented reality via the identified user account;

storing the augmented reality in conjunction with the user account on a server;

determining that the user account, identified based on the optical barcode, is associated with the augmented reality;

retrieving the augmented reality in response to determining that the user account, identified based on the optical barcode, is associated with the augmented reality; and

transmitting, to the client device from the server, the retrieved augmented reality in response to determining that the user account, identified based on the optical barcode, is associated with the augmented reality.

5. The method of claim 4 , further comprising generating a user interface item to include an interface for connecting with the user account via an instant messaging application.

6. The method of claim 5 , further comprising generating the user interface item to include a link to a webpage associated with or provided by the user account.

7. The method of claim 1 , the augmented reality being transmitted to the client device for display within the custom graphic of the image.

8. The method of claim 1 further comprising:

generating a low resolution version of the image;

modifying the low resolution version of the image;

extracting the candidate shape features from the modified low resolution version of the image; and

generating a high resolution version of a particular portion of the image corresponding to the extracted candidate shape features.

9. The method of claim 1 , wherein the custom graphic is included in a portion of the image and comprises a first region bounded by a second region, wherein the optical barcode is in a portion of the custom graphic between the first and second regions and is excluded from the first region, and wherein the first region includes visual content visible that is part of the image received from the client device.

10. The method of claim 1 further comprising aligning the custom graphic based on a distinctive feature of the custom graphic.

11. A non-transitory machine-readable medium comprising instructions which, when executed by one or more hardware processors of a client device, cause the client device to implement operations comprising:

identifying candidate shape features in an image;

identifying a custom graphic by comparing the candidate shape features with a reference shape feature of the custom graphic, the comparing comprising:

computing an area score by comparing a first candidate shape feature of the candidate shape features that represents an area value with a reference area of the reference shape;

determining that the area score fails to exceed an area rule threshold; and

analyzing a second candidate shape feature of the candidate shape features to identify the custom graphic in response to determining that the area score fails to exceed the area rule threshold; and

identifying an optical barcode based on the identified custom graphic;

transmitting the image to a server based on identification of the optical barcode;

receiving, from the server, an augmented reality associated with the optical barcode; and

displaying the augmented reality on an electronic display of the client device;

transmitting an indication to the server that the image including the optical barcode was selected;

receiving, from the server, a user interface item associated with a user account associated with the optical barcode in response to the transmitting of the indication; and

displaying the user interface item on the electronic display.

12. The machine-readable medium of claim 11 , further comprising receiving an input indicating the selection was of a predetermined shape within the image.

13. The machine-readable medium of claim 11 , the operations further comprising:

receiving a selection of the user interface item; and

carrying out an action corresponding to the selection of the user interface item.

14. The machine-readable medium of claim 11 , the operations further comprising capturing the image via a camera of the client device.

15. The machine-readable medium of claim 14 , the operations further comprising:

identifying a user account of an instant messaging application based on the optical barcode; and

capturing the image via the instant messaging application of the client device.

16. The machine-readable medium of claim 15 , the operations further comprising playing a video indicated by the augmented reality.

17. The machine-readable medium of claim 16 , wherein the user interface item comprises an interface for connecting with the user account via the instant messaging application, the operations further comprising connecting with the user account of the instant messaging application via the interface, and playing the video based on the connection.

18. The machine-readable medium of claim 17 , wherein the user interface item comprises a link to a webpage associated with or provided by the user account, and wherein the connecting with the user account is based on the link.

19. The machine-readable medium of claim 18 , the operations further comprising displaying at least a portion of the image external to the custom graphic while displaying the augmented reality within the custom graphic of the image.

20. The machine-readable medium of claim 19 , wherein the portion of the image comprises the optical barcode.

21. A system comprising:

one or more hardware processors; and

a memory storing instructions which, when executed by the one or more hardware processors, cause the one or more hardware processors to perform operations comprising:

identifying candidate shape features from an image;

identifying a custom graphic by comparing the candidate shape features with a reference shape feature of the custom graphic, the comparing comprising:

computing an area score by comparing a first candidate shape feature of the candidate shape features that represents an area value with a reference area of the reference shape;

determining that the area score fails to exceed an area rule threshold; and

analyzing a second candidate shape feature of the candidate shape features to identify the custom graphic in response to determining that the area score fails to exceed the area rule threshold; and

identifying an optical barcode based on the identified custom graphic;

identifying an augmented reality based on the optical barcode; and

transmitting, by the one or more hardware processors, the augmented reality to the client device;

identifying, by the one or more hardware processors, a user account based on the optical barcode;

receiving, by the one or more hardware processors, an indication that the image including the optical barcode was selected at the client device; and

transmitting, to the client device, a user interface item associated with the user account in response to the indication.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 21, 2019
From: BOYD, NATHAN KENNETH; LUC, CHRISTOPHER JONATHAN; OUIMET, KIRK; SHETH, RAHUL BHUPENDRA
To: SNAPCHAT, INC.
Reel/Frame 049553/0224 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 21, 2019
From: SNAPCHAT, INC.
To: SNAP INC.
Reel/Frame 049553/0327 →
Continuity (1)
Continuation 15262942 · Sep 12, 2016
Cited By (3)
US 12,301,941 US 12,341,738 US 12,393,324