IP Library Granted Patent US 12,475,158
Granted Patent B2
US 12,475,158 · App. 17/499,680 · Granted Nov 18, 2025

Context based media curation

Inventors: Kaveh Anvaripour (Santa Monica, CA); Ebony James Charlton (Los Angeles, CA); Travis Chen (Los Angeles, CA); Celia Nicole Mourkogiannis (Los Angeles, CA); Kevin Dechau Tang (Los Angeles, CA)
Assignee: Snap Inc.
G06F16/434G06F16/24578G06F16/41G06F16/438G06F16/44G06F16/48G06K7/1413G06K7/1417G06N20/00H04L51/10H04L51/212G06F3/0482
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,475,158
App. No.
17/499,680
Granted
Nov 18, 2025
Kind
B2
Abstract

A media curation system configured to perform operations that include, capturing an image at a client device, wherein the image includes a depiction of an object, identifying an object category of the object based on the depiction of the object within the image, accessing media content associated with the object category within a media repository, generating a presentation of the media content, and causing display of the presentation of the media content within the image at the client device.

Claims (63)

1 . A method comprising:

causing display of an image that comprises a plurality of image features within a graphical user interface (GUI) at a client device, the GUI including a menu element that includes a graphical icon;

receiving, at the client device, an input that selects the graphical icon from within the menu element;

accessing user profile data associated with a user of the client device, the user profile data including user affinities;

determining an input context of the client device based on the user profile data responsive to the input that selects the graphical icon;

identifying a first object and a second object depicted by the image based on the plurality of image features of the image responsive to the input that selects the graphical icon from within the menu element, the first object corresponding with a first media tag, and the second object corresponding with a second media tag;

causing display of an indicator corresponding with one or more of the first object, the second object, or the input context;

generating a query to a media repository based on at least the input context that includes the user affinities from the user profile data, the first media tag and the second media tag;

accessing a set of media items from within the media repository based on the query, the set of media items comprising tags that correspond with object categories and input contexts;

determining a ranking the set of media items based on the tags, the query, and the input context; and

causing display of a presentation of the set of media items from the media repository at the client device based on the ranking, the presentation of the set of media items comprising a navigable arrangement of a set of icons displayed horizontally within the GUI of the client device.

2 . The method of claim 1 , wherein the causing display of the presentation of the set of media items further comprises:

causing display of a notification in response to the accessing the set of media items from within the repository;

receiving a selection of the notification; and

causing display of the presentation of the set of media items in response to the selection of the notification.

3 . The method of claim 2 , wherein the notification comprises a display of an indication of a number of media items among the set of media items.

4 . The method of claim 1 , wherein the identifying the object depicted by the image based on the plurality of image features further comprises:

detecting the plurality of image features; and

presenting an indicator at a position within the image in response to the detecting the plurality of image features.

5 . The method of claim 4 , wherein the indicator comprises a graphical property based on the category of the object.

6 . The method of claim 1 , wherein the media overlay comprises AR content to be presented at the client device.

7 . The method of claim 1 , wherein the determining an input context of the client device includes accessing a user profile data, the user profile data comprising one or more of a user device type, a user affinity, or a network connection of the client device.

8 . A system comprising:

a memory; and

at least one hardware processor coupled to the memory and comprising instructions that causes the system to perform operations comprising:

causing display of an image that comprises a plurality of image features within a graphical user interface (GUI) at a client device, the GUI including a menu element that includes a graphical icon;

receiving, at the client device, an input that selects the graphical icon from within the menu element;

accessing user profile data associated with a user of the client device, the user profile data including user affinities;

determining an input context of the client device based on the user profile data responsive to the input that selects the graphical icon;

identifying a first object and a second object depicted by the image based on the plurality of image features of the image responsive to the input that selects the graphical icon from within the menu element, the first object corresponding with a first media tag, and the second object corresponding with a second media tag;

causing display of an indicator corresponding with one or more of the first object, the second object, or the input context;

generating a query to a media repository based on at least the input context that includes the user affinities from the user profile data, the first media tag and the second media tag;

accessing a set of media items from within the media repository based on the query, the set of media items comprising tags that correspond with object categories and input contexts;

determining a ranking the set of media items based on the tags, the query, and the input context; and

causing display of a presentation of the set of media items from the media repository at the client device based on the ranking, the presentation of the set of media items comprising a navigable arrangement of a set of icons displayed horizontally within the GUI of the client device.

9 . The system of claim 8 , wherein the causing display of the presentation of the set of media items further comprises:

causing display of a notification in response to the accessing the set of media items from within the repository;

receiving a selection of the notification; and

causing display of the presentation of the set of media items in response to the selection of the notification.

10 . The system of claim 9 , wherein the notification comprises a display of an indication of a number of media items among the set of media items.

11 . The system of claim 8 , wherein the identifying the object depicted by the image based on the plurality of image features further comprises:

detecting the plurality of image features; and

presenting an indicator at a position within the image in response to the detecting the plurality of image features.

12 . The system of claim 11 , wherein the indicator comprises a graphical property based on the category of the object.

13 . A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

causing display of an image that comprises a plurality of image features within a graphical user interface (GUI) at a client device, the GUI including a menu element that includes a graphical icon;

receiving, at the client device, an input that selects the graphical icon from within the menu element;

accessing user profile data associated with a user of the client device, the user profile data including user affinities;

determining an input context of the client device based on the user profile data responsive to the input that selects the graphical icon;

identifying a first object and a second object depicted by the image based on the plurality of image features of the image responsive to the input that selects the graphical icon from within the menu element, the first object corresponding with a first media tag, and the second object corresponding with a second media tag;

causing display of an indicator corresponding with one or more of the first object, the second object, or the input context;

generating a query to a media repository based on at least the input context that includes the user affinities from the user profile data, the first media tag and the second media tag;

accessing a set of media items from within the media repository based on the query, the set of media items comprising tags that correspond with object categories and input contexts;

determining a ranking the set of media items based on the tags, the query, and the input context; and

causing display of a presentation of the set of media items from the media repository at the client device based on the ranking, the presentation of the set of media items comprising a navigable arrangement of a set of icons displayed horizontally within the GUI of the client device.

14 . The non-transitory machine-readable storage medium of claim 13 , wherein the causing display of the presentation of the set of media items further comprises:

causing display of a notification in response to the accessing the set of media items from within the repository;

receiving a selection of the notification; and

causing display of the presentation of the set of media items in response to the selection of the notification.

15 . The non-transitory machine-readable storage medium of claim 14 , wherein the notification comprises a display of an indication of a number of media items among the set of media items.

16 . The non-transitory machine-readable storage medium of claim 13 , wherein the identifying the object depicted by the image based on the plurality of image features further comprises:

detecting the plurality of image features; and

presenting an indicator at a position within the image in response to the detecting the plurality of image features.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 19, 2021
From: ANVARIPOUR, KAVEH; CHARLTON, EBONY JAMES; CHEN, TRAVIS; MOURKOGIANNIS, CELIA NICOLE; TANG, KEVIN DECHAU
To: SNAP INC.
Reel/Frame 057835/0895 →
Continuity (2)
Continuation 16370373 · Mar 29, 2019
Related Publication 20220036079A1 · Feb 3, 2022
References Cited (89)
US 6804683B1 · Matsuzaki · 2004 [cited by examiner]
US 7455240B2 · Chapman, Jr. · 2008 [cited by examiner]
US 8175847B2 · Zhang et al. · 2012 [cited by applicant]
US 8498490B1 · Baluja et al. · 2013 [cited by applicant]
US 9142253B2 · Ubillos · 2015 [cited by examiner]
US 9342930B1 · Kraft et al. · 2016 [cited by applicant]
US 9349095B1 · Tseng et al. · 2016 [cited by applicant]
US 10733255B1 · Yang · 2020 [cited by examiner]
US 10795567B2 · Vegesna · 2020 [cited by examiner]
US 10820030B2 · Nishimura · 2020 [cited by examiner]
US 11024099B1 · Little · 2021 [cited by examiner]
US 11170035B2 · Anvaripour et al. · 2021 [cited by applicant]
US 20060123360A1 · Anwar · 2006 [cited by examiner]
US 20080021928A1 · Yagnik · 2008 [cited by applicant]
US 20080147664A1 · Fujiwara · 2008 [cited by examiner]
US 20080240494A1 · Oosawa · 2008 [cited by examiner]
US 20080256450A1 · Takakura · 2008 [cited by examiner]
US 20090116752A1 · Isomura · 2009 [cited by examiner]
US 20090172598A1 · Yamanaka · 2009 [cited by examiner]
US 20100011392A1 · Bronstein et al. · 2010 [cited by applicant]
US 20100058216A1 · Yoon · 2010 [cited by examiner]
US 20110029635A1 · Shkurko · 2011 [cited by examiner]
US 20110081948A1 · Shirai et al. · 2011 [cited by applicant]
US 20110096093A1 · Oi et al. · 2011 [cited by applicant]
US 20110099478A1 · Gallagher et al. · 2011 [cited by applicant]
US 20110246182A1 · Allen · 2011 [cited by examiner]
US 20110314031A1 · Chittar et al. · 2011 [cited by applicant]
US 20120251011A1 · Gao et al. · 2012 [cited by applicant]
US 20130148864A1 · Dolson · 2013 [cited by examiner]
US 20130169853A1 · Luong · 2013 [cited by applicant]
US 20140022265A1 · Canan et al. · 2014 [cited by applicant]
US 20140320516A1 · Son · 2014 [cited by examiner]
US 20140365496A1 · Chung · 2014 [cited by examiner]
US 20150169994A1 · Chinen · 2015 [cited by examiner]
US 20150202962A1 · Habashima et al. · 2015 [cited by applicant]
US 20150317365A1 · Andress · 2015 [cited by examiner]
US 20150317380A1 · Rathnavelu et al. · 2015 [cited by applicant]
US 20160034459A1 · Larsen · 2016 [cited by examiner]
US 20160162482A1 · Pesavento et al. · 2016 [cited by applicant]
US 20160283595A1 · Folkens et al. · 2016 [cited by applicant]
US 20160350421A1 · Multerer · 2016 [cited by examiner]
US 20170011121A1 · Martin · 2017 [cited by examiner]
US 20170083795A1 · Ono · 2017 [cited by applicant]
US 20170206431A1 · Sun et al. · 2017 [cited by applicant]
US 20170323185A1 · Bhardwaj et al. · 2017 [cited by applicant]
US 20170351710A1 · Zhu et al. · 2017 [cited by applicant]
US 20170357433A1 · Boule · 2017 [cited by examiner]
US 20180097988A1 · Tsuchiya · 2018 [cited by examiner]
US 20180101894A1 · Markoff · 2018 [cited by examiner]
US 20180129724A1 · Heine · 2018 [cited by examiner]
US 20180225290A1 · Leppänen et al. · 2018 [cited by applicant]
US 20180336182A1 · Lachmi et al. · 2018 [cited by applicant]
US 20190147054A1 · Behnken et al. · 2019 [cited by applicant]
US 20200311116A1 · Anvaripour et al. · 2020 [cited by applicant]
CN 108369583A · 2018 [cited by applicant]
CN 113892095A · 2022 [cited by applicant]
KR 20150034724A · 2015 [cited by applicant]
KR 20160041085A · 2016 [cited by applicant]
WO WO2006062772A1 · 2006 [cited by applicant]
WO WO2007103804A2 · 2007 [cited by applicant]
WO WO2012142332A1 · 2012 [cited by applicant]
WO WO2015038515A1 · 2015 [cited by examiner]
WO WO2016070098A2 · 2016 [cited by applicant]
WO WO2016018785A1 · 2017 [cited by examiner]
WO WO2017048359A1 · 2017 [cited by examiner]
WO WO2017147305A1 · 2017 [cited by examiner]
WO WO2018104834A1 · 2018 [cited by examiner]
WO WO2018217008A1 · 2018 [cited by applicant]
WO WO2020205436A1 · 2020 [cited by applicant]
Ruoqian Liu et al., “PinterNet: A thematic label curation tool for large image datasets”, 2016 IEEE International Conference on Big Data (Big Data), Dec. 2016, pp. 2353-2362. [cited by examiner]
Lasse Wollatz et al., “Curation of Image Data for Medical Research”, 2018 IEEE 14th International Conference on e-Science (e-Science),Oct.-Nov. 2018, pp. 105-113. [cited by examiner]
Feng Zhao et al., “An Intelligent and Interactive Icon Creation System for Mobile Users”, 2010 Fifth International Conference on Systems, Apr. 2010, pp. 1-5. [cited by examiner]
“U.S. Appl. No. 16/370,373, Final Office Action mailed Apr. 28, 2021”, 15 pgs. [cited by applicant]
“U.S. Appl. No. 16/370,373, Non Final Office Action mailed Mar. 10, 2021”, 14 pgs. [cited by applicant]
“U.S. Appl. No. 16/370,373, Notice of Allowance mailed Jul. 14, 2021”, 21 pgs. [cited by applicant]
“U.S. Appl. No. 16/370,373, Response filed Apr. 19, 2021 to Non Final Office Action mailed Mar. 10, 2021”, 10 pgs. [cited by applicant]
“U.S. Appl. No. 16/370,373, Response filed Jun. 16, 2021 to Final Office Action mailed Apr. 28, 2021”, 11 pgs. [cited by applicant]
“International Application Serial No. PCT/US2020/024994, International Search Report mailed Jul. 6, 2020”, 3 pgs. [cited by applicant]
“International Application Serial No. PCT/US2020/024994, Written Opinion mailed Jul. 6, 2020”, 7 pgs. [cited by applicant]
Sandnes, Frode Eika, et al., “Where was that photo taken? Deriving geographical information from image collections based on temporal exposure attributes”, Multimedia Systems, 16, (2010), 309-318. [cited by applicant]
Wang, Xiaogang, et al., “Query-specific visual semantic spaces for web image re-ranking”, IEEE, (Jun. 20-25, 2011), 857-864. [cited by applicant]
“European Application Serial No. 20720273.0, Communication Pursuant to Article 94(3) EPC mailed Feb. 24, 2023”, 7 pgs. [cited by applicant]
“International Application Serial No. PCT/US2020/024994, International Preliminary Report on Patentability mailed Oct. 14, 2021”, 9 pgs. [cited by applicant]
“European Application Serial No. 20720273.0, Summons to Attend Oral Proceedings mailed Dec. 22, 2023”, 12 pgs. [cited by applicant]
“European Application Serial No. 20720273.0, Response filed Apr. 11, 2024 to Summons to Attend Oral Proceedings mailed Dec. 22, 2023”, 12 pgs. [cited by applicant]
“Korean Application Serial No. 10-2021-7034749, Notice of Preliminary Rejection mailed Apr. 17, 2024”, w/ English translation, 18 pgs. [cited by applicant]
“Korean Application Serial No. 10-2021-7034749, Response filed Jun. 12, 2024 to Notice of Preliminary Rejection mailed Apr. 17, 2024”, w/ English Claims, 31 pgs. [cited by applicant]
“European Application Serial No. 20720273.0, EPO Written Decision to Refuse mailed Jun. 17, 2024”, 23 pgs. [cited by applicant]
“Chinese Application Serial No. 202080039248.2, Office Action mailed Apr. 25, 2025”, w/ English Translation, 12 pgs. [cited by applicant]