IP Library › Granted Patent US 12,322,046
Granted Patent B2
US 12,322,046 · App. 18/499,811 · Granted Jun 3, 2025

Augmented reality content generators for spatially browsing travel destinations

Inventors: Ilteris Kaan Canberk (Marina Del Rey, CA); Virginia Drummond (Venice, CA); Jean Luo (Seattle, WA); Alek Matthiessen (Marina Del Rey, CA); Celia Nicole Mourkogiannis (Los Angeles, CA)
Assignee: Snap Inc.
G06T19/003G06F3/04817G06F3/0488G06T19/006
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,322,046
App. No.
18/499,811
Filed
Nov 1, 2023
Granted
Jun 3, 2025
Kind
B2
Art Unit
2615
USPC
345/633
Abstract

The subject technology selects a set of augmented reality content generators from a plurality of augmented reality content generators. The subject technology causes display, at a client device, of a graphical interface comprising a plurality of selectable graphical items. The subject technology receives, at the client device, a selection of a first selectable graphical item from the plurality of selectable graphical items, the first selectable graphical item comprising a first augmented reality content generator corresponding to a particular geolocation. The subject technology causes display, at the client device, at least one augmented content reality item generated by the first augmented reality content generator. The subject technology causes display, at a second client device, the at least one augmented content reality item generated by the first augmented reality content generator, the at least one augmented content reality item comprising the 3D scene based at least in part on the particular geolocation.

Claims (50)

1. A method, comprising:

receiving, by a messaging client application at a client device, a selection of a first selectable graphical item from a plurality of selectable graphical items, the first selectable graphical item comprising a first augmented reality content generator corresponding to a particular geolocation, the particular geolocation being different than a current geolocation of the client device, the client device comprising a processor, the processor executing the messaging client application;

after receiving the selection, causing display, by the messaging client application executing on the processor at the client device, a plurality of graphical items corresponding to a plurality of users, each graphical item comprising a name of a particular user from the plurality of users as an indication of the particular user to participate in a shared augmented reality viewing experience provided by the first selectable graphical item corresponding to an augmented reality content generator that has been selected;

causing display, by the messaging client application executing on the processor at the client device, at least one augmented reality content item generated by the first augmented reality content generator, the at least one augmented reality content item comprising a 3D scene based at least in part on the particular geolocation different than the current geolocation of the client device;

causing display, by a second messaging client application at a second client device, the at least one augmented reality content item generated by the first augmented reality content generator, the at least one augmented reality content item comprising the 3D scene based at least in part on the particular geolocation, the second client device comprising a particular processor executing the second messaging client application, the second client device being communicatively coupled to the client device over a network;

causing display, by the messaging client application at the client device, of a graphical interface comprising a plurality of selectable graphical items, each selectable graphical item corresponding to a respective augmented reality content generator of a set of augmented reality content generators; and

causing display of respective 3D objects in a particular arrangement, each 3D object corresponding to one of the plurality of selectable graphical items, wherein the respective 3D objects are displayed in a current scene captured by a camera of the client device.

2. The method of claim 1 , wherein the particular geolocation is different than a second current geolocation of the second client device, and further comprising:

receiving a touch input via a touch screen of the client device; and

in response to receiving the touch input, causing navigation through the 3D scene.

3. The method of claim 2 , wherein the touch input comprises a swipe gesture.

4. The method of claim 1 , wherein the particular arrangement comprises a geometric arrangement where each 3D object is positioned at an equal distance in 3D space from at least one other 3D object.

5. The method of claim 1 , wherein the particular arrangement comprises a geometric arrangement where each 3D object is positioned at different distances in 3D space from at least a first 3D object and a second 3D object.

6. The method of claim 1 , further comprising:

receiving movement data from the client device, the movement data indicating that the current scene captured by the camera has changed; and

in response to receiving the movement data, rendering an updated view of the current scene based on the received movement data, the updated view of the current scene reflecting the changed current scene.

7. The method of claim 6 , further comprising:

causing display of the respective 3D objects in the updated view of the current scene.

8. The method of claim 1 , wherein the second client device is associated with a different user.

9. The method of claim 1 , further comprising:

receiving movement data from the second client device, the movement data indicating that the 3D scene based at least in part on the particular geolocation different than the current geolocation of the client device has changed; and

in response to receiving the movement data, causing an updated view of the 3D scene to be rendered at the client device based on the received movement data from the second client device.

10. A system comprising:

a processor; and

a memory including instructions that, when executed by the processor, cause the processor to perform operations comprising:

receiving, by a messaging client application at a client device, a selection of a first selectable graphical item from a plurality of selectable graphical items, the first selectable graphical item comprising a first augmented reality content generator corresponding to a particular geolocation, the particular geolocation being different than a current geolocation of the client device, the client device comprising a processor, the processor executing the messaging client application;

after receiving the selection, causing display, by the messaging client application executing on the processor at the client device, a plurality of graphical items corresponding to a plurality of users, each graphical item comprising a name of a particular user from the plurality of users as an indication of the particular user to participate in a shared augmented reality viewing experience provided by the first selectable graphical item corresponding to an augmented reality content generator that has been selected;

causing display, by the messaging client application executing on the processor at the client device, at least one augmented reality content item generated by the first augmented reality content generator, the at least one augmented reality content item comprising a 3D scene based at least in part on the particular geolocation different than the current geolocation of the client device;

causing display, by a second messaging client application at a second client device, the at least one augmented reality content item generated by the first augmented reality content generator, the at least one augmented reality content item comprising the 3D scene based at least in part on the particular geolocation, the second client device comprising a particular processor executing the second messaging client application, the second client device being communicatively coupled to the client device over a network;

causing display, by the messaging client application at the client device, of a graphical interface comprising a plurality of selectable graphical items, each selectable graphical item corresponding to a respective augmented reality content generator of a set of augmented reality content generators; and

causing display of respective 3D objects in a particular arrangement, each 3D object corresponding to one of the plurality of selectable graphical items, wherein the respective 3D objects are displayed in a current scene captured by a camera of the client device.

11. The system of claim 10 , wherein the particular geolocation is different than a second current geolocation of the second client device, and wherein the operations further comprise:

receiving a touch input via a touch screen of the client device; and

in response to receiving the touch input, causing navigation through the 3D scene.

12. The system of claim 11 , wherein the touch input comprises a swipe gesture.

13. The system of claim 10 , wherein the particular arrangement comprises a geometric arrangement where each 3D object is positioned at an equal distance in 3D space from at least one other 3D object.

14. The system of claim 10 , wherein the particular arrangement comprises a geometric arrangement where each 3D object is positioned at different distances in 3D space from at least a first 3D object and a second 3D object.

15. The system of claim 10 , wherein the operations further comprise:

receiving movement data from the client device, the movement data indicating that the current scene captured by the camera has changed; and

in response to receiving the movement data, rendering an updated view of the current scene based on the received movement data, the updated view of the current scene reflecting the changed current scene.

16. The system of claim 15 , wherein the operations further comprise:

causing display of the respective 3D objects in the updated view of the current scene.

17. The system of claim 10 , wherein the second client device is associated with a different user.

18. A non-transitory computer-readable medium comprising instructions, which when executed by a computing device, cause the computing device to perform operations comprising:

receiving, by a messaging client application at a client device, a selection of a first selectable graphical item from a plurality of selectable graphical items, the first selectable graphical item comprising a first augmented reality content generator corresponding to a particular geolocation, the particular geolocation being different than a current geolocation of the client device, the client device comprising a processor, the processor executing the messaging client application;

after receiving the selection, causing display, by the messaging client application executing on the processor at the client device, a plurality of graphical items corresponding to a plurality of users, each graphical item comprising a name of a particular user from the plurality of users as an indication of the particular user to participate in a shared augmented reality viewing experience provided by the first selectable graphical item corresponding to an augmented reality content generator that has been selected;

causing display, by the messaging client application executing on the processor at the client device, at least one augmented reality content item generated by the first augmented reality content generator, the at least one augmented reality content item comprising a 3D scene based at least in part on the particular geolocation different than the current geolocation of the client device;

causing display, by a second messaging client application at a second client device, the at least one augmented reality content item generated by the first augmented reality content generator, the at least one augmented reality content item comprising the 3D scene based at least in part on the particular geolocation, the second client device comprising a particular processor executing the second messaging client application, the second client device being communicatively coupled to the client device over a network;

causing display, by the messaging client application at the client device, of a graphical interface comprising a plurality of selectable graphical items, each selectable graphical item corresponding to a respective augmented reality content generator of a set of augmented reality content generators; and

causing display of respective 3D objects in a particular arrangement, each 3D object corresponding to one of the plurality of selectable graphical items, wherein the respective 3D objects are displayed in a current scene captured by a camera of the client device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 1, 2023
From: CANBERK, ITERIS KAAN; DRUMMOND, VIRGINIA; LUO, JEAN; MATTHIESSEN, ALEK; MOURKOGIANNIS, CELIA NICOLE
To: SNAP INC.
Reel/Frame 065424/0501 →
Continuity (3)
Continuation 17235280 · Apr 20, 2021
Provisional Application 63085470 · Sep 30, 2020
Related Publication 20240062474A1 · Feb 22, 2024
References Cited (143)
US 8681178B1 · Tseng · 2014 [cited by applicant]
US 8797353B2 · Bregman-amitai et al. · 2014 [cited by applicant]
US 9632671B2 · Albright · 2017 [cited by applicant]
US 10242477B1 · Charlton et al. · 2019 [cited by applicant]
US 10484817B1 · Chipouras et al. · 2019 [cited by applicant]
US 10496272B1 · Lonkar · 2019 [cited by examiner]
US 10559107B1 · Charlton et al. · 2020 [cited by applicant]
US 10740974B1 · Cowburn et al. · 2020 [cited by applicant]
US 10789473B2 · Jung et al. · 2020 [cited by applicant]
US 10839219B1 · Suiter et al. · 2020 [cited by applicant]
US 10956743B1 · Li et al. · 2021 [cited by applicant]
US 11044393B1 · Suiter et al. · 2021 [cited by applicant]
US 11201981B1 · Suiter et al. · 2021 [cited by applicant]
US 11270067B1 · Paul · 2022 [cited by applicant]
US 11538225B2 · Anvaripour et al. · 2022 [cited by applicant]
US 11592311B2 · Yoo et al. · 2023 [cited by applicant]
US 11625142B2 · An et al. · 2023 [cited by applicant]
US 11809507B2 · Anvaripour et al. · 2023 [cited by applicant]
US 11816805B2 · Anvaripour et al. · 2023 [cited by applicant]
US 11836826B2 · Canberk et al. · 2023 [cited by applicant]
US 12039499B2 · Anvaripour et al. · 2024 [cited by applicant]
US 20080097698A1 · Arnold-Huyser et al. · 2008 [cited by applicant]
US 20120001939A1 · Sandberg · 2012 [cited by applicant]
US 20120075341A1 · Sandberg · 2012 [cited by applicant]
US 20130332890A1 · Ramic et al. · 2013 [cited by applicant]
US 20140076965A1 · Becorest et al. · 2014 [cited by applicant]
US 20150127486A1 · Advani · 2015 [cited by examiner]
US 20150206349A1 · Rosenthal et al. · 2015 [cited by applicant]
US 20160047672A1 · Sachdev et al. · 2016 [cited by applicant]
US 20160133230A1 · Daniels · 2016 [cited by examiner]
US 20160308985A1 · Qi et al. · 2016 [cited by applicant]
US 20170103442A1 · Wright · 2017 [cited by examiner]
US 20170123614A1 · Perlegos et al. · 2017 [cited by applicant]
US 20170199496A1 · Grata et al. · 2017 [cited by applicant]
US 20170371883A1 · Bailiang et al. · 2017 [cited by applicant]
US 20180082430A1 · Sharma et al. · 2018 [cited by applicant]
US 20180167692A1 · Kedenburg, III et al. · 2018 [cited by applicant]
US 20180191797A1 · Javier et al. · 2018 [cited by applicant]
US 20180224560A1 · Xi et al. · 2018 [cited by applicant]
US 20180278462A1 · Bjontegard · 2018 [cited by applicant]
US 20180315134A1 · Amitay et al. · 2018 [cited by applicant]
US 20180322174A1 · Vasilyev et al. · 2018 [cited by applicant]
US 20180349413A1 · Shelby et al. · 2018 [cited by applicant]
US 20190005739A1 · Gazlay et al. · 2019 [cited by applicant]
US 20190018859A1 · Miyaoka · 2019 [cited by applicant]
US 20190073831A1 · Kim · 2019 [cited by applicant]
US 20190179947A1 · O'brien et al. · 2019 [cited by applicant]
US 20190279250A1 · Gordon et al. · 2019 [cited by applicant]
US 20190295056A1 · Wright · 2019 [cited by applicant]
US 20190310757A1 · Lee et al. · 2019 [cited by applicant]
US 20190318540A1 · Piemonte et al. · 2019 [cited by applicant]
US 20200066044A1 · Stahl et al. · 2020 [cited by applicant]
US 20200128106A1 · McCormack · 2020 [cited by examiner]
US 20200220823A1 · Eirinberg et al. · 2020 [cited by applicant]
US 20200357165A1 · Dixit et al. · 2020 [cited by applicant]
US 20210042724A1 · Rathod · 2021 [cited by applicant]
US 20210056761A1 · Nigam et al. · 2021 [cited by applicant]
US 20210097875A1 · Alexander, IV et al. · 2021 [cited by applicant]
US 20220004308A1 · Simpson · 2022 [cited by applicant]
US 20220100812A1 · Anvaripour et al. · 2022 [cited by applicant]
US 20220101565A1 · Anvaripour et al. · 2022 [cited by applicant]
US 20220101606A1 · Canberk et al. · 2022 [cited by applicant]
US 20220101610A1 · Anvaripour et al. · 2022 [cited by applicant]
US 20230079325A1 · Anvaripour et al. · 2023 [cited by applicant]
US 20240061898A1 · Anvaripour et al. · 2024 [cited by applicant]
US 20240062493A1 · Anvaripour et al. · 2024 [cited by applicant]
US 20240320631A1 · Anvaripour et al. · 2024 [cited by applicant]
CN 107038616 · 2017 [cited by applicant]
CN 110709835 · 2020 [cited by applicant]
CN 116210019A · 2023 [cited by applicant]
CN 116261710A · 2023 [cited by applicant]
CN 116325663A · 2023 [cited by applicant]
CN 116964547 · 2023 [cited by applicant]
CN 116210019B · 2024 [cited by applicant]
JP 2022121496A · 2022 [cited by applicant]
KR 20110134736A · 2011 [cited by applicant]
KR 20130140339 · 2013 [cited by applicant]
KR 20150037725 · 2015 [cited by applicant]
KR 102635705 · 2024 [cited by applicant]
WO 2020069401 · 2020 [cited by applicant]
WO WO2021076754A1 · 2021 [cited by applicant]
WO WO2022072177A1 · 2022 [cited by applicant]
WO WO2022072187A2 · 2022 [cited by applicant]
WO WO2022072187A3 · 2022 [cited by applicant]
WO WO2022072976A1 · 2022 [cited by applicant]
WO WO2022072983A1 · 2022 [cited by applicant]
“U.S. Appl. No. 17/146,973, Non Final Office Action mailed Apr. 5, 2022”, 13 pgs. [cited by applicant]
“U.S. Appl. No. 17/146,973, Notice of Allowance mailed Aug. 22, 2022”, 7 pgs. [cited by applicant]
“U.S. Appl. No. 17/146,973, Response filed Jul. 5, 2022 to Non Final Office Action mailed Apr. 5, 2022”, 11 pgs. [cited by applicant]
“U.S. Appl. No. 17/147,008, Final Office Action mailed Aug. 23, 2022”, 14 pgs. [cited by applicant]
“U.S. Appl. No. 17/147,008, Non Final Office Action mailed Feb. 9, 2023”, 14 pgs. [cited by applicant]
“U.S. Appl. No. 17/147,008, Non Final Office Action mailed Mar. 17, 2022”, 13 pgs. [cited by applicant]
“U.S. Appl. No. 17/147,008, Notice of Allowance mailed Jun. 27, 2023”, 8 pgs. [cited by applicant]
“U.S. Appl. No. 17/147,008, Response filed Jan. 20, 2023 to Final Office Action mailed Aug. 23, 2022”, 11 pgs. [cited by applicant]
“U.S. Appl. No. 17/147,008, Response filed May 9, 2023 to Non Final Office Action mailed Feb. 9, 2023”, 12 pgs. [cited by applicant]
“U.S. Appl. No. 17/147,008, Response filed Jun. 17, 2022 to Non Final Office Action mailed Mar. 17, 2022”, 10 pgs. [cited by applicant]
“U.S. Appl. No. 17/235,280, Final Office Action mailed Jan. 26, 2023”, 20 pgs. [cited by applicant]
“U.S. Appl. No. 17/235,280, Final Office Action mailed Feb. 4, 2022”, 16 pgs. [cited by applicant]
“U.S. Appl. No. 17/235,280, Non Final Office Action mailed Aug. 10, 2022”, 21 pgs. [cited by applicant]
“U.S. Appl. No. 17/235,280, Non Final Office Action mailed Sep. 22, 2021”, 14 pgs. [cited by applicant]
“U.S. Appl. No. 17/235,280, Notice of Allowance mailed Jul. 11, 2023”, 14 pgs. [cited by applicant]
“U.S. Appl. No. 17/235,280, Response filed Jan. 24, 2022 to Non Final Office Action mailed Sep. 22, 2021”, 12 pgs. [cited by applicant]
“U.S. Appl. No. 17/235,280, Response filed May 4, 2022 to Final Office Action mailed Feb. 4, 2022”, 13 pgs. [cited by applicant]
“U.S. Appl. No. 17/235,280, Response filed May 26, 2023 to Final Office Action mailed Jan. 26, 2023”, 12 pgs. [cited by applicant]
“U.S. Appl. No. 17/235,280, Response filed Nov. 10, 2022 to Non Final Office Action mailed Aug. 10, 2022”, 13 pgs. [cited by applicant]
“U.S. Appl. No. 17/235,411, Non Final Office Action mailed Aug. 31, 2023”, 12 pgs. [cited by applicant]
“U.S. Appl. No. 18/056,122, Corrected Notice of Allowability mailed Aug. 4, 2023”, 2 pgs. [cited by applicant]
“U.S. Appl. No. 18/056,122, Non Final Office Action mailed Mar. 15, 2023”, 15 pgs. [cited by applicant]
“U.S. Appl. No. 18/056,122, Notice of Allowance mailed Jul. 10, 2023”, 7 pgs. [cited by applicant]
“U.S. Appl. No. 18/056,122, Response filed Jun. 13, 2023 to Non Final Office Action mailed Mar. 15, 2023”, 9 pgs. [cited by applicant]
“Chinese Application Serial No. 202180066305.0, Notification to Make Rectification mailed Jun. 2, 2023”, W/O English Translation, 1 page. [cited by applicant]
“Chinese Application Serial No. 202180066305.0, Notification to Make Rectification mailed Sep. 5, 2023”, W/O English Translation, 1 page. [cited by applicant]
“Chinese Application Serial No. 202180066475.9, Voluntary Amendment filed Sep. 21, 2023”, W/English Translation, 12 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/035389, International Search Report mailed Sep. 27, 2021”, 3 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/035389, Written Opinion mailed Sep. 27, 2021”, 4 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/051294, International Preliminary Report on Patentability mailed Apr. 13, 2023”, 8 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/051294, International Search Report mailed Dec. 22, 2021”, 5 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/051294, Written Opinion mailed Dec. 22, 2021”, 6 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/051419, International Preliminary Report on Patentability mailed Apr. 13, 2023”, 11 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/051419, International Search Report mailed Mar. 21, 2022”, 7 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/051419, Invitation to Pay Additional Fees mailed Jan. 7, 2022”, 10 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/051419, Written Opinion mailed Mar. 21, 2022”, 9 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/071493, International Preliminary Report on Patentability mailed Apr. 13, 2023”, 9 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/071493, International Search Report mailed Dec. 6, 2021”, 4 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/071493, Written Opinion mailed Dec. 6, 2021”, 7 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/071570, International Preliminary Report on Patentability mailed Apr. 13, 2023”, 9 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/071570, International Search Report mailed Dec. 23, 2021”, 5 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/071570, Written Opinion mailed Dec. 23, 2021”, 7 pgs. [cited by applicant]
“Publish 3D scene”, Adobe Dimension User Guide, XP055873491, [Online] Retrieved from the Internet: <URL: https://helpx.adobe.com/dimension/using/publish-3d-scene.html>, (Jun. 12, 2020), 3 pgs. [cited by applicant]
Brata, KC Liang, et al., “Location-based augmented reality information for bus route planning system”, International Journal of Electrical and Computer Engineering, (Feb. 2015), 8 pgs. [cited by applicant]
Du, Ruofei, et al., “Social street view: blending immersive street views with geo-tagged social media”, ACM WEB3D Technology, [Online] Retrieved from the Internet: <URL: https://dl.acm.org/doi/pdf/10.145/2945292.2945294… [cited by applicant]
Nayyar, A Mahapatra, et al., “Virtual Reality (VR) & Augmented Reality (AR) technologies for tourism and hospitality industry”, International journal of engineering &technology., (2018), 156-60. [cited by applicant]
Neuburger, Larissa Julia, et al., “The ‘Phygital’tourist experience: The use of augmented and virtual reality in destination marketing”, in Tourism planning and destination marketing, Emerald Publishing Limited, (2018),… [cited by applicant]
“U.S. Appl. No. 18/499,872, Non Final Office Action mailed Jul. 16, 2024”, 9 pgs. [cited by applicant]
“U.S. Appl. No. 18/501,875, Non Final Office Action mailed Jun. 20, 2024”, 18 pgs. [cited by applicant]
“U.S. Appl. No. 18/501,875, Response filed Sep. 19, 2024 to Non Final Office Action mailed Jun. 20, 2024”, 9 pgs. [cited by applicant]
“U.S. Appl. No. 17/235,280, Corrected Notice of Allowability mailed Nov. 8, 2023”, 11 pgs. [cited by applicant]
“U.S. Appl. No. 17/235,411, Response filed Jan. 2, 2024 to Non Final Office Action mailed Aug. 31, 2023”, 14 pgs. [cited by applicant]
“Chinese Application Serial No. 202180066475.9, Office Action mailed Jan. 26, 2024”, w/ English Translation, 11 pgs. [cited by applicant]
“U.S. Appl. No. 17/235,411, Notice of Allowance mailed Mar. 4, 2024”, 8 pgs. [cited by applicant]
“U.S. Appl. No. 17/235,411, Corrected Notice of Allowability mailed Mar. 25, 2024”, 2 pgs. [cited by applicant]
“Korean Application Serial No. 10-2023-7014503, Notice of Preliminary Rejection mailed Apr. 3, 2024”, w/ English Translation, 10 pgs. [cited by applicant]
“Chinese Application Serial No. 202180066475.9, Response filed May 13, 2024 to Office Action mailed Jan. 26, 2024”, 51 pgs. [cited by applicant]
Cited By (1)
US 12,561,651