IP Library Granted Patent US 11,017,233
Granted Patent B2
US 11,017,233 · App. 16/814,221 · Granted May 25, 2021

Contextual media filter search

Inventors: Ebony James Charlton (Los Angeles, CA); Hao Hu (Bellevue, WA); Yanjia Li (Torrance, CA); Xing Mei (Los Angeles, CA); Kevin Dechau Tang (Los Angeles, CA)
Assignee: Snap Inc.
G06K9/00671G06F3/0488G06F16/51G06F16/538G06T7/60G06T11/00G06T2207/30242
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,017,233
App. No.
16/814,221
Granted
May 25, 2021
Kind
B2
Abstract

Method for receiving an input onto a graphical user interface at a client device, capturing an image frame at the client device, the image frame comprising a depiction of an object, identifying the object within the image frame, accessing media content associated with the object within a media repository in response to identifying the object, and causing presentation of the media content within the image frame at the client device.

Claims (72)

1. A computer-implemented method comprising:

receiving, using one or more processors, an input onto a graphical user interface at a client device;

in response to the input, capturing, using the one or more processors, an image frame at the client device, the image frame comprising a depiction of an object;

identifying, using the one or more processors, the object within the image frame, the identifying the object within the image frame comprising:

identifying a first set of image features within the image frame;

storing the first set of features in an indexing structure;

identifying a second set of image features based on the first set of image features, the identifying the second set of image features comprising:

mapping each feature in the first set of image features to a representative feature in a set of representative features; and

counting an occurrence of each representative feature in the set of representative features;

searching the indexing structure using the second set of image features;

accessing, using the one or more processors, media content associated with the object within a media repository in response to identifying the object within the image frame; and

causing, using the one or more processors, presentation of the media content within the image frame at the client device.

2. The method of claim 1 , wherein receiving the input onto the graphical user interface at the client device further comprises:

receiving an initiation of the input comprising a tactile input onto the graphical user interface of the client device;

detecting that a property of the tactile input transgresses a threshold value; and

capturing the image frame in response to the property of the tactile input transgressing the threshold value.

3. The method of claim 2 , wherein the property of the tactile input includes one or more of:

an input pressure; and

an input duration.

4. The method of claim 1 , wherein the first set of features are local features identified within predefined regions within the image frame.

5. The method of claim 1 , wherein the second set of image features are global features identified using an entirety of the image frame.

6. The method of claim 1 , wherein accessing the media content associated with the object further comprises:

verifying the media content using geometric verification;

based on the geometric verification, identifying false media content in the media content; and

removing the false media content from the media content.

7. The method of claim 1 , wherein the media content associated with the object is visually similar to the object, based on a similarity metric.

8. The method of claim 1 , wherein the image frame is received from a messaging application on the client device.

9. The method of claim 1 , wherein the media content comprises a plurality of augmented reality experience selections.

10. The method of claim 9 , wherein the plurality of augmented reality experience selections is associated with a plurality of special effects, respectively.

11. A system comprising:

a processor; and

a memory storing instructions that, when executed by the processor, configure the processor to perform operations comprising:

receiving an input onto a graphical user interface at a client device;

in response to the input, capturing an image frame at the client device, the image frame comprising a depiction of an object;

identifying the object within the image frame, the identifying the object within the image frame comprising:

identifying a first set of image features within the image frame;

storing the first set of features in an indexing structure;

identifying a second set of image features based on the first set of image features, the identifying the second set of image features comprising:

mapping each feature in the first set of image features to a representative feature in a set of representative features; and

counting an occurrence of each representative feature in the set of representative features;

searching the indexing structure using the second set of image features;

accessing media content associated with the object within a media repository in response to identifying the object within the image frame; and

causing presentation of the media content within the image frame at the client device.

12. The system of claim 11 , wherein receiving the input onto the graphical user interface at the client device further comprises:

receiving an initiation of the input comprising a tactile input onto the graphical user interface of the client device;

detecting that a property of the tactile input transgresses a threshold value; and

capturing the image frame in response to the property of the tactile input transgressing the threshold value.

13. The system of claim 12 , wherein the property of the tactile input includes one or more of:

an input pressure; and

an input duration.

14. The system of claim 11 , wherein the first set of features are local features identified within predefined regions within the image frame.

15. The system of claim 11 , wherein the second set of image features are global features identified using an entirety of the image frame.

16. The system of claim 11 , wherein accessing the media content associated with the object further comprises:

verifying the media content using geometric verification;

based on the geometric verification, identifying false media content in the media content; and

removing the false media content from the media content.

17. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

receiving an input onto a graphical user interface at a client device;

in response to the input, capturing an image frame at the client device, the image frame comprising a depiction of an object;

identifying the object within the image frame, the identifying the object within the image frame comprising:

identifying a first set of image features within the image frame;

storing the first set of features in an indexing structure;

identifying a second set of image features based on the first set of image features, the identifying the second set of image features comprising:

mapping each feature in the first set of image features to a representative feature in a set of representative features; and

counting an occurrence of each representative feature in the set of representative features;

searching the indexing structure using the second set of image features;

accessing media content associated with the object within a media repository in response to identifying the object within the image frame; and

causing presentation of the media content within the image frame at the client device.

18. The non-transitory machine-readable storage medium of claim 17 , wherein receiving the input onto the graphical user interface at the client device further comprises:

receiving an initiation of the input comprising a tactile input onto the graphical user interface of the client device;

detecting that a property of the tactile input transgresses a threshold value; and

capturing the image frame in response to the property of the tactile input transgressing the threshold value.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 26, 2021
From: CHARLTON, EBONY JAMES; HU, HAO; LI, YANJIA; MEI, XING; TANG, KEVIN DECHAU
To: SNAP INC.
Reel/Frame 056039/0738 →
Continuity (2)
Provisional Application 62826679 · Mar 29, 2019
Related Publication 20200311426A1 · Oct 1, 2020
Cited By (5)
US 12,277,632 US 12,315,495 US 12,361,934 US 12,393,734 US 12,399,927