IP Library › Granted Patent US 12,292,299
Granted Patent B2
US 12,292,299 · App. 18/318,401 · Granted May 6, 2025

Augmented reality-based translations associated with travel

Inventors: Virginia Drummond (Venice, CA); Jean Luo (Seattle, WA); Alek Matthiessen (Marina Del Rey, CA); Celia Nicole Mourkogiannis (Los Angeles, CA)
Assignee: Snap Inc.
G01C21/3614G01C21/3476G06F3/017G06V20/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,292,299
App. No.
18/318,401
Filed
May 16, 2023
Granted
May 6, 2025
Kind
B2
Examiner
SITTA, GRANT
Art Unit
2622
USPC
701/533
Abstract

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing augmented reality content corresponding to a translation in association with travel. The program and method provide for receiving, by a messaging application running on a device of a user, a request to scan an image captured by a device camera; obtaining, in response to receiving the request, a travel parameter associated with the request, and an attribute of an object depicted in the image; determining, based on the travel parameter and the attribute, to perform a translation with respect to the object; performing, in response to the determining, the translation with respect to the object; and displaying an augmented reality content item, which includes the translation, with the image.

Claims (44)

1. A method, comprising:

receiving, by a messaging application running on a device of a user, a request to scan an image captured by a device camera;

obtaining, in response to receiving the request, a travel parameter associated with the request, and an attribute of an object depicted in the image;

determining, based on the travel parameter and the attribute, to perform a translation with respect to the object;

performing, in response to the determining, the translation with respect to the object;

displaying an augmented reality content item, which includes the translation, with the image;

receiving, during display of the translation, user input corresponding to a tilt gesture of the device; and

performing, in response to receiving the user input, a second translation with respect to the object.

2. The method of claim 1 , wherein the travel parameter indicates at least one of a travel schedule, a transportation schedule, a language, a general location, a specific venue or landmark, an activity, a list of participants, or a topic of interest.

3. The method of claim 1 , wherein the attribute comprises a name of the object, and

wherein the translation is performed with respect to the name of the object.

4. The method of claim 1 , wherein the attribute comprises machine-encoded text based on text depicted within the image, and

wherein the translation is performed with respect to the machine-encoded text.

5. The method of claim 1 , wherein determining to perform the translation is further based on a geolocation of the device.

6. The method of claim 1 , wherein the image corresponds to a live feed of a camera of the device.

7. The method of claim 1 , wherein the image corresponds to a saved image included in a photo library.

8. A system, comprising:

at least one processor; and

a memory storing instructions that, when executed by the at least one processor, configure the processor to perform operations comprising:

receiving, by a messaging application running on a device of a user, a request to scan an image captured by a device camera;

obtaining, in response to receiving the request, a travel parameter associated with the request, and an attribute of an object depicted in the image;

determining, based on the travel parameter and the attribute, to perform a translation with respect to the object;

performing, in response to the determining, the translation with respect to the object;

displaying an augmented reality content item, which includes the translation, with the image;

receiving, during display of the translation, user input corresponding to a tilt gesture of the device; and

performing, in response to receiving the user input, a second translation with respect to the object.

9. The system of claim 8 , wherein the travel parameter indicates at least one of a travel schedule, a transportation schedule, a language, a general location, a specific venue or landmark, an activity, a list of participants, or a topic of interest.

10. The system of claim 8 , wherein the attribute comprises a name of the object, and wherein the translation is performed with respect to the name of the object.

11. The system of claim 8 , wherein the attribute comprises machine-encoded text based on text depicted within the image, and wherein the translation is performed with respect to the machine-encoded text.

12. The system of claim 8 , wherein determining to perform the translation is further based on a geolocation of the device.

13. The system of claim 8 , wherein the image corresponds to a live feed of a camera of the device.

14. The system of claim 8 , wherein the image corresponds to a saved image included in a photo library.

15. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to:

receiving, by a messaging application running on a device of a user, a request to scan an image captured by a device camera;

obtaining, in response to receiving the request, a travel parameter associated with the request, and an attribute of an object depicted in the image;

determining, based on the travel parameter and the attribute, to perform a translation with respect to the object;

performing, in response to the determining, the translation with respect to the object;

displaying an augmented reality content item, which includes the translation, with the image;

receiving, during display of the translation, user input corresponding to a tilt gesture of the device; and

performing, in response to receiving the user input, a second translation with respect to the object.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the travel parameter indicates at least one of a travel schedule, a transportation schedule, a language, a general location, a specific venue or landmark, an activity, a list of participants, or a topic of interest.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the attribute comprises a name of the object, and wherein the translation is performed with respect to the name of the object.

18. The non-transitory computer-readable storage medium of claim 15 , wherein the attribute comprises machine-encoded text based on text depicted within the image, and

wherein the translation is performed with respect to the machine-encoded text.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 16, 2023
From: DRUMMOND, VIRGINIA; LUO, JEAN; MATTHIESSEN, ALEK; MOURKOGIANNIS, CELIA NICOLE
To: SNAP INC.
Reel/Frame 063657/0975 →
Continuity (3)
Continuation 17155819 · Jan 22, 2021
Provisional Application 62705505 · Jun 30, 2020
Related Publication 20230280176A1 · Sep 7, 2023
References Cited (28)
US 10387730B1 · Cowburn et al. · 2019 [cited by applicant]
US 10529139B1 · Greene · 2020 [cited by applicant]
US 11308327B2 · Drummond et al. · 2022 [cited by applicant]
US 11526368B2 · Karashchuk · 2022 [cited by examiner]
US 11680814B2 · Drummond · 2023 [cited by examiner]
US 20090063129A1 · Tsai et al. · 2009 [cited by applicant]
US 20150309997A1 · Lee · 2015 [cited by examiner]
US 20150370785A1 · Mauser · 2015 [cited by examiner]
US 20150371407A1 · Kim et al. · 2015 [cited by applicant]
US 20180052832A1 · Anglin et al. · 2018 [cited by applicant]
US 20180129276A1 · Nguyen et al. · 2018 [cited by applicant]
US 20180210874A1 · Fuxman · 2018 [cited by examiner]
US 20180253901A1 · Charlton et al. · 2018 [cited by applicant]
US 20180300916A1 · Barnett et al. · 2018 [cited by applicant]
US 20190025909A1 · Mittal · 2019 [cited by examiner]
US 20190379617A1 · Luo et al. · 2019 [cited by applicant]
US 20210404831A1 · Drummond et al. · 2021 [cited by applicant]
US 20220179665A1 · Rathod · 2022 [cited by applicant]
CN 115836292A · 2023 [cited by applicant]
WO WO2022005843A1 · 2022 [cited by applicant]
“U.S. Appl. No. 17/155,819, Non Final Office Action filed Aug. 1, 2022”, 15 pgs. [cited by applicant]
“U.S. Appl. No. 17/155,819, Notice of Allowance filed Feb. 7, 2023”, 10 pgs. [cited by applicant]
“U.S. Appl. No. 17/155,819, Response filed Nov. 1, 2022 to Non Final Office Action filed Aug. 1, 2022”, 7 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/038684, International Preliminary Report on Patentability mailed Jan. 12, 2023”, 9 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/038684, International Search Report mailed Oct. 5, 2021”, 4 pgs. [cited by applicant]
“International Application Serial No. PCT/US2021/038684, Written Opinion mailed Oct. 5, 2021”, 7 pgs. [cited by applicant]
U.S. Appl. No. 17/155,819, filed Jan. 22, 2021, Augmented Reality-Based Translations Associated With Travel. [cited by applicant]
“European Application Serial No. 21742275.7, Response filed Aug. 7, 2023 to Communication pursuant to Rules 161(1) and 162 EPC mailed Feb. 7, 2023”, 18 pgs. [cited by applicant]