IP Library › Granted Patent US 12,323,478
Granted Patent B2
US 12,323,478 · App. 18/329,954 · Granted Jun 3, 2025

Triggering changes to real-time special effects included in a live streaming video

Inventors: Artem Gaiduchenko (Marina de Rey, CA); Artem Yerofieiev (Marina del Rey, CA); Bohdan Pozharskyi (Venice, CA); Gabriel Lupin (Marina del Rey, CA); Oleksii Kholovchuk (Marina del Rey, CA); Travis Chen (Los Angeles, CA); Yurii Monastyrshyn (Santa Monica, CA); Denys Makoviichuk (Marina Del Rey, CA)
Assignee: Snap Inc.
H04L65/762H04L65/61
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,323,478
App. No.
18/329,954
Granted
Jun 3, 2025
Kind
B2
Abstract

Method for triggering changes to real-time special effects included in a live streaming video starts with a processor transmitting in real-time a video stream captured by a camera via a network. The processor causes a live streaming interface that includes the video stream to be displayed on the plurality of client devices. The processor receives a trigger to apply one of a plurality of special effects to the video stream and determines a first special effect of the plurality of special effects is associated with the trigger. The processor applies in real-time the first special effect to the video stream to generate a video stream having the first special effect and transmits in real-time the video stream having the first special effect via the network. The processor causes the live streaming interface that includes the video stream having the first special effect to be displayed on the plurality of client devices. Other embodiments are disclosed.

Claims (65)

1. A system comprising:

at least one processor; and

a memory component having instructions stored thereon, when executed by the at least one processor, causes the at least one processor to perform operations comprising:

presenting an augmented reality experience selection user interface comprising a plurality of selectable thumbnails representing respective augmented reality experiences;

presenting a visual indicator in association with an individual selectable thumbnail of the plurality of thumbnails, the visual indicator visually distinguishing the individual selectable thumbnail from other selectable thumbnails in the plurality of selectable thumbnails, the visual indicator comprising text suggesting to a user to set an individual augmented reality experience corresponding to the individual selectable thumbnail as a preferred augmented reality experience;

receiving input from the user that sets one or more augmented reality experiences comprising the individual augmented reality experience, from a plurality of augmented reality experiences, as preferred augmented reality experiences by selecting one or more selectable thumbnails from the plurality of selectable thumbnails presented in the augmented reality experience selection;

presenting a live video captured by a camera on a display;

presenting a tray icon associated with a set of augmented reality experiences; and

in response to receiving input that selects the tray icon, overlaying, on top of the live video, a tray that includes the one or more augmented reality experiences that previously were set by the received input as the preferred augmented reality experiences.

2. The system of claim 1 , the operations comprising:

causing a live streaming interface to be displayed on a plurality of devices, the live streaming interface including a video stream and a live stream of a video game being played by the user in real-time, the video stream and the live stream of the video game being displayed independently in the live streaming interface;

receiving a trigger to apply one of a plurality of special effects to the video stream, the plurality of special effects comprising one of the one or more augmented reality experiences that previously were set by the received input as the preferred augmented reality experiences; and

applying in real-time a first augmented reality experience of the plurality of augmented reality experiences to the video stream.

3. The system of claim 1 , the operations comprising:

setting a subset of the plurality of augmented reality experiences as the preferred augmented reality experiences, the subset comprising the one or more augmented reality experiences; and

receiving additional input from the user that customizes an order in which the subset of the plurality of augmented reality experiences appears in the tray.

4. The system of claim 1 , each of the selectable thumbnails comprising previous of a special effect associated with the respective augmented reality experience.

5. The system of claim 2 , wherein receiving the trigger includes:

receiving a selection of a display item included in the live streaming interface from one of the plurality of devices.

6. The system of claim 2 , the operations further comprising:

receiving a customized setting that associates a display item with the first augmented reality experience.

7. The system of claim 2 , wherein receiving the trigger includes:

receiving an input from the user via a lens selection user interface selecting a first lens selection.

8. The system of claim 1 , the operations comprising:

receiving input form a user setting a key sequence comprising a plurality of keys for a first lens selection;

storing the key sequence in association with the first lens selection;

receiving an input from the user of a set of keys on a keyboard corresponding to the key sequence; and

triggering presentation of the first lens selection in response to determining that the set of keys in the received input correspond to the stored key sequence.

9. The system of claim 2 , the operations further comprising:

receiving a subsequent trigger to apply one of the plurality of augmented reality experiences to the video stream;

determining a second augmented reality experience of the plurality of augmented reality experiences is associated with the subsequent trigger; and

applying in real-time the second augmented reality experience to the video stream to generate a video stream having the second augmented reality experience.

10. The system of claim 2 , wherein receiving the trigger includes:

receiving the trigger from a third-party application.

11. The system of claim 10 , wherein:

the third-party application is an email application, a messaging application, or a telephone application, and

the trigger being generated based on an occurrence within the third-party application comprising a new email notification, a new message notification, or a telephone call notification.

12. The system of claim 11 , wherein the third-party application is the video game, wherein the occurrence within the video game includes a predetermined action by or on a user's character.

13. A method comprising:

presenting an augmented reality experience selection user interface comprising a plurality of selectable thumbnails representing respective augmented reality experiences;

presenting a visual indicator in association with an individual selectable thumbnail of the plurality of thumbnails, the visual indicator visually distinguishing the individual selectable thumbnail from other selectable thumbnails in the plurality of selectable thumbnails, the visual indicator comprising text suggesting to a user to set an individual augmented reality experience corresponding to the individual selectable thumbnail as a preferred augmented reality experience;

receiving input from the user that sets one or more augmented reality experiences comprising the individual augmented reality experience, from a plurality of augmented reality experiences, as preferred augmented reality experiences by selecting one or more selectable thumbnails from the plurality of selectable thumbnails presented in the augmented reality experience selection;

presenting a live video captured by a camera on a display;

presenting a tray icon associated with a set of augmented reality experiences; and

in response to receiving input that selects the tray icon, overlaying, on top of the live video, a tray that includes the one or more augmented reality experiences that previously were set by the received input as the preferred augmented reality experiences.

14. The method of claim 13 , further comprising:

causing a live streaming interface to be displayed on a plurality of devices, the live streaming interface including a video stream and a live stream of a video game being played by a user in real-time, the video stream and the live stream of the video game being displayed independently in the live streaming interface;

receiving a trigger to apply one of a plurality of special effects to the video stream, the plurality of special effects comprising one of the one or more augmented reality experiences that previously were set by the received input as the preferred augmented reality experiences; and

applying in real-time a first augmented reality experience of the plurality of augmented reality experiences to the video stream.

15. The method of claim 14 , further comprising:

transmitting in real-time the video stream having the first augmented reality experience via a network.

16. The method of claim 14 , further comprising causing the live streaming interface including the video stream having the first augmented reality experience to be displayed on the plurality of devices.

17. The method of claim 14 , wherein receiving the trigger includes:

receiving a selection of a display item included in the live streaming interface from one of the plurality of devices.

18. The method of claim 14 , further comprising:

receiving a customized setting that associates a display item with the first augmented reality experience.

19. The method of claim 14 , wherein receiving the trigger includes:

receiving an input from the user via a lens selection user interface selecting a first lens selection.

20. A non-transitory computer storage medium, having stored thereon instructions, when executed by at least one processor, causes the at least one processor to perform operations comprising:

presenting an augmented reality experience selection user interface comprising a plurality of selectable thumbnails representing respective augmented reality experiences;

presenting a visual indicator in association with an individual selectable thumbnail of the plurality of thumbnails, the visual indicator visually distinguishing the individual selectable thumbnail from other selectable thumbnails in the plurality of selectable thumbnails, the visual indicator comprising text suggesting to a user to set an individual augmented reality experience corresponding to the individual selectable thumbnail as a preferred augmented reality experience;

receiving input from the user that sets one or more augmented reality experiences comprising the individual augmented reality experience, from a plurality of augmented reality experiences, as preferred augmented reality experiences by selecting one or more selectable thumbnails from the plurality of selectable thumbnails presented in the augmented reality experience selection;

presenting a live video captured by a camera on a display;

presenting a tray icon associated with a set of augmented reality experiences; and

in response to receiving input that selects the tray icon, overlaying, on top of the live video, a tray that includes the one or more augmented reality experiences that previously were set by the received input as the preferred augmented reality experiences.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 6, 2023
From: GAIDUCHENKO, ARTEM; YEROFIEIEV, ARTEM; POZHARSKYI, BOHAN; LUPIN, GABRIEL; KHOLOVCHUK, OLEKSII; CHEN, TRAVIS; MONASTYRSHYN, YURII; MAKOVIICHUK, DENYS
To: SNAP INC.
Reel/Frame 063868/0357 →
Continuity (4)
Continuation 17538545 · Nov 30, 2021
Continuation 16264515 · Jan 31, 2019
Provisional Application 62751365 · Oct 26, 2018
Related Publication 20230319126A1 · Oct 5, 2023
References Cited (35)
US 6317127B1 · Daily et al. · 2001 [cited by applicant]
US 9754397B1 · Piemonte et al. · 2017 [cited by applicant]
US 10498794B1 · Keighran et al. · 2019 [cited by applicant]
US 10569174B1 · Wheeler et al. · 2020 [cited by applicant]
US 11212331B1 · Gaiduchenko et al. · 2021 [cited by applicant]
US 20070040850A1 · Coleman · 2007 [cited by applicant]
US 20130281208A1 · Lyons et al. · 2013 [cited by applicant]
US 20140281014A1 · Good et al. · 2014 [cited by applicant]
US 20140347265A1 · Aimone et al. · 2014 [cited by applicant]
US 20150334347A1 · Kang et al. · 2015 [cited by applicant]
US 20170228130A1 · Palmaro · 2017 [cited by examiner]
US 20170324624A1 · Taine · 2017 [cited by examiner]
US 20180068490A1 · Holmes · 2018 [cited by applicant]
US 20180077469A1 · Geduldig · 2018 [cited by applicant]
US 20180098059A1 · Valdivia · 2018 [cited by examiner]
US 20180167427A1 · Kedenburg, III · 2018 [cited by examiner]
US 20180250589A1 · Cossairt et al. · 2018 [cited by applicant]
US 20190132650A1 · Kedenburg, III · 2019 [cited by examiner]
US 20190228580A1 · Pelant et al. · 2019 [cited by applicant]
US 20190261025A1 · Hussain · 2019 [cited by examiner]
US 20190318543A1 · Skidmore et al. · 2019 [cited by applicant]
US 20220166816A1 · Gaiduchenko et al. · 2022 [cited by applicant]
U.S. Appl. No. 16/264,515 U.S. Pat. No. 11,212,331, filed Jan. 31, 2019, Triggering Changes to Real-Time Special Effects Included in a Live Streaming Video. [cited by applicant]
U.S. Appl. No. 17/538,545, filed Nov. 30, 2021, Triggering Changes to Real-Time Special Effects Included in a Live Streaming Video. [cited by applicant]
“U.S. Appl. No. 16/264,515, Final Office Action mailed Oct. 7, 2020”, 10 pgs. [cited by applicant]
“U.S. Appl. No. 16/264,515, Non Final Office Action mailed Jan. 19, 2021”, 10 pgs. [cited by applicant]
“U.S. Appl. No. 16/264,515, Non Final Office Action mailed Apr. 15, 2020”, 9 pgs. [cited by applicant]
“U.S. Appl. No. 16/264,515, Notice of Allowance mailed Aug. 18, 2021”, 8 pgs. [cited by applicant]
“U.S. Appl. No. 16/264,515, Response filed Jan. 7, 2021 to Final Office Action mailed Oct. 7, 2020”, 11 pgs. [cited by applicant]
“U.S. Appl. No. 16/264,515, Response filed May 19, 2021 to Non Final Office Action mailed Jan. 19, 2021”, 11 pgs. [cited by applicant]
“U.S. Appl. No. 16/264,515, Response filed Jul. 15, 2020 to Non Final Office Action mailed Apr. 15, 2020”, 10 pgs. [cited by applicant]
“U.S. Appl. No. 17/538,545, Non Final Office Action mailed Sep. 19, 2022”, 10 pgs. [cited by applicant]
“U.S. Appl. No. 17/538,545, Notice of Allowance mailed Mar. 6, 2023”, 8 pgs. [cited by applicant]
“U.S. Appl. No. 17/538,545, Preliminary Amendment filed Sep. 19, 2022”, 7 pgs. [cited by applicant]
“U.S. Appl. No. 17/538,545, Response filed Nov. 18, 2022 to Non Final Office Action mailed Sep. 19, 2022”, 9 pgs. [cited by applicant]