IP Library › Granted Patent US 12,477,163
Granted Patent B2
US 12,477,163 · App. 18/326,724 · Granted Nov 18, 2025

Machine learning model continuous training system

Inventors: Kevin Sarabia Dela Rosa (Seattle, WA); Hao Hu (Bellevue, WA); Yanjia Li (Torrance, CA)
Assignee: Snap Inc.
H04N21/251G06V10/774G06V20/41G06V20/46H04N21/23418
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,477,163
App. No.
18/326,724
Granted
Nov 18, 2025
Kind
B2
Abstract

Described is a system for performing a set of machine learning model training operations that include: accessing media content items associated with interaction functions initiated by users of an interaction system, generating training data including labels for the media content items, extracting features from a media content item of the media content items, identifying additional media content items to include in the training data based on the extracted features from the media content item, processing the training data using a machine learning model to generate a media content item output; and updating one or more parameters of the machine learning model based on the media content item output. The system checks whether retraining criteria has been met, and repeats the set of machine learning model training operations to retrain the machine learning model.

Claims (46)

1 . A system comprising:

at least one processor;

at least one memory component storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:

training a machine learning model by performing a set of operations comprising:

accessing media content items associated with interaction functions initiated by users of an interaction system, wherein the media content items comprise images, videos, or content augmentations of the users posted on the interaction system enabling other users to view the posted media content items;

generating training data including labels for the media content items, wherein the labels are indicative of one or more characteristics of the media content items;

extracting features from a media content item of the media content items;

identifying additional media content items to include in the training data based on the extracted features from the media content item;

processing the training data using a machine learning model to generate a media content item output; and

updating one or more parameters of the machine learning model based on the media content item output; and

repeating the set of operations to retrain the machine learning model based on a retraining criterion being met.

2 . The system of claim 1 , wherein identifying additional media content items comprises applying a distance metric to compare the media content item and individual additional media content items in order to identify the additional media content items.

3 . The system of claim 2 , wherein extracting the features from the media content item comprises applying a machine learning model trained to extract features from one or more media content items.

4 . The system of claim 3 , wherein generating the training data comprises adding the one or more extracted features to the labels.

5 . The system of claim 2 , wherein the media content items comprise videos created by users to share with other users, wherein the features are extracted on a frame-by-frame basis.

6 . The system of claim 2 , wherein the media content items comprise content augmentations created by users to share with other users, wherein the extracted features include the augmentations that are applied to a camera feed in real-time.

7 . The system of claim 2 , wherein at least some of the media content items are in a different format than the additional media content items.

8 . The system of claim 7 , wherein the media content items that are in the different format are compared with the additional media contents based on the extracted features.

9 . The system of claim 1 , wherein the additional media content items were created by users in a different time period than when the accessed media content items were created.

10 . The system of claim 1 , wherein the additional media content items were created by different users than the users that created the accessed media content items.

11 . The system of claim 1 , wherein the additional media content items are identified based on metadata of the accessed media content items.

12 . The system of claim 11 , wherein the metadata comprises a location where a user created the individual media content item.

13 . The system of claim 1 , wherein the retraining criterion comprises a keyword indicative of a trend, wherein the operations further comprise tracking the use of one or more keywords in media content items and the retraining criterion includes meeting a threshold number of uses of the keyword.

14 . The system of claim 1 , wherein the media content items comprise images or videos, and the interaction functions comprise media content items created by users and shared with other users.

15 . The system of claim 14 , wherein the media content items that were created by users do not include labels for training the machine learning model, wherein generating the training data includes identifying keywords in captions of individual media content items or comments to the media content items from other users.

16 . The system of claim 14 , wherein the media content items comprise content augmentations that add interactive digital elements in real-time to a camera feed.

17 . The system of claim 1 , wherein the operations further comprise adding the training data to an existing set of training data, wherein repeating the set of operations further comprises adding newly accessed training data to the existing set of training data such that the existing set of training data increases in size with each repeating of the set of operations.

18 . The system of claim 17 , wherein processing the training data using the machine learning model to generate the media content item output further comprises processing the existing set of training data using the machine learning model.

19 . A method comprising:

training a machine learning model by performing a set of operations comprising:

accessing media content items associated with interaction functions initiated by users of an interaction system, wherein the media content items comprise images, videos, or content augmentations of the users posted on the interaction system enabling other users to view the posted media content items;

generating training data including labels for the media content items, wherein the labels are indicative of one or more characteristics of the media content items;

extracting features from a media content item of the media content items;

identifying additional media content items to include in the training data based on the extracted features from the media content item;

processing the training data using a machine learning model to generate a media content item output; and

updating one or more parameters of the machine learning model based on the media content item output; and

repeating the set of operations to retrain the machine learning model based on a retraining criterion being met.

20 . A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

training a machine learning model by performing a set of operations comprising:

accessing media content items associated with interaction functions initiated by users of an interaction system, wherein the media content items comprise images, videos, or content augmentations of the users posted on the interaction system enabling other users to view the posted media content items;

generating training data including labels for the media content items, wherein the labels are indicative of one or more characteristics of the media content items;

extracting features from a media content item of the media content items;

identifying additional media content items to include in the training data based on the extracted features from the media content item;

processing the training data using a machine learning model to generate a media content item output; and

updating one or more parameters of the machine learning model based on the media content item output; and

repeating the set of operations to retrain the machine learning model based on a retraining criterion being met.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 31, 2023
From: SARABIA DELA ROSA, KEVIN; HU, HAO; LI, YANJIA
To: SNAP INC.
Reel/Frame 064438/0429 →
Continuity (1)
Related Publication 20240406477A1 · Dec 5, 2024
References Cited (29)
US 10410138B2 · Maor et al. · 2019 [cited by applicant]
US 20120208166A1 · Ernst et al. · 2012 [cited by applicant]
US 20170262633A1 · Miserendino et al. · 2017 [cited by applicant]
US 20190258900A1 · Baikadi et al. · 2019 [cited by applicant]
US 20200012962A1 · Dent et al. · 2020 [cited by applicant]
US 20210012239A1 · Arzani · 2021 [cited by examiner]
US 20210027182A1 · Harris et al. · 2021 [cited by applicant]
US 20220121955A1 · Chavoshi et al. · 2022 [cited by applicant]
US 20220230089A1 · Peraud et al. · 2022 [cited by applicant]
US 20240428470A1 · Dela Rosa et al. · 2024 [cited by applicant]
WO 2018141942 · 2018 [cited by applicant]
“AWS re:Invent 2019: Build accurate training datasets with Amazon SageMaker Ground Truth (AIM308)”, [Online] Retrieved from the Internet: <URL: https://www.youtube.com/watch?v=6WJxzKsIFKA>, (Dec. 5, 2019), 1 pg. [cited by applicant]
Hu, Derek Hao, “Towards Continuous Computer Vision Model Improvement with Kubeflow”, [Online] Retrieved from the Internet: <URL: https://www.youtube.com/watch?v=9UPnCo-LG04>, (Nov. 22, 2019), 1 pg. [cited by applicant]
“International Application Serial No. PCT/US2024/034814, International Search Report mailed Sep. 13, 2024”, 3 pgs. [cited by applicant]
“International Application Serial No. PCT/US2024/034814, Written Opinion mailed Sep. 13, 2024”, 8 pgs. [cited by applicant]
“Can you create a Bitmoji from a picture?—Quora”, XP093200757, [Online]. Retrieved from the Internet: <https://www.quora.com/Can-you- create- a-Bitmoji-from-a-picture/answer/Clossen-Gabrial>, (2021), 1 pg. [cited by applicant]
“Augmented Reality (AR) tutorial for beginners using Unity 2022”, XP093200647, [Online]. Retrieved from the Internet: <https://www.youtube.com/watch?v=gpaq5bAjya8>, (Jun. 27, 2022), 2 pgs. [cited by applicant]
“Algorithm—Image similarity comparison—Stack Overflow”, [Online]. Retrieved from the Internet: <https://stackoverflow.com/questions/5730631/image-similarity-comparison>, (Jun. 28, 2017), 5 pgs. [cited by applicant]
“International Application Serial No. PCT/US2024/031880, International Search Report mailed Oct. 16, 2024”, 3 pgs. [cited by applicant]
“International Application Serial No. PCT/US2024/031880, Written Opinion mailed Oct. 16, 2024”, 13 pgs. [cited by applicant]
Eskenazi, Cecile, “Snapchat's 3D animated Bitmoji avatar in the real-world, via Snapchat's AR camera”, XP093200748, [Online]. Retrieved from the Internet: <https://vnm.youtube.com/shorts/uvgQ-JsUZ1c>, (Sep. 23, 2017), 1… [cited by applicant]
Hu, Benyi, “PyRetri: A PyTorch-based Library for Unsupervised Image Retrieval by Deep Convolutional Neural Networks”, arXiv:2005.02154v2 [cs.IR], (Aug. 5, 2020), 4 pgs. [cited by applicant]
Ko, Byungsoo, “A Benchmark on Tricks for Large-scale Image Retrieval”, arXiv:1907.11854v2 [cs.CV], (Apr. 23, 2020), 9 pgs. [cited by applicant]
Mahajan, Dhruv, “Exploring the Limits of Weakly Supervised Pretraining”, arXiv:1805.00932v1 [cs.CV], (May 2, 2018), 23 pgs. [cited by applicant]
Matsui, Yusuke, “Billion-scale Approximate Nearest Neighbor Search”, CVPR 2020 Tutorial, [Online]. Retrieved from the Internet: <URL: https://www.youtube.com/watch?v=SKrHs03i08Q&list=PLKQB14e0EJUWaTnwgQogJ3nSLzEFNn9d8&t… [cited by applicant]
Nace, Trevor, “How to Create Snapchat Bitmoji With Camera”, XP093200719, [Online]. Retrieved from the Internet: <URL: https://www.youtube.com/watch?v=Ldz5wUzHSUk>>, (Dec. 5, 2023), 2 pgs. [cited by applicant]
Radenovic, Filip, “Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking”, arXiv:1803.11285v1 [cs.CV], (Mar. 29, 2018), 10 pgs. [cited by applicant]
Tan, Mingxing, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks”, arXiv:1905.11946v5 [cs. LG], (2019), 11 pgs. [cited by applicant]
Web Whizz, “How to Scan Face for Bitmoji on Snapchat 2024”, XP093200775, [Online]. Retrieved from the Internet: <URL: https://www.youtube.com/watch7v- HRb3UmrW91, (Aug. 26, 2022), 2 pgs. [cited by applicant]
Cited By (1)
US 12,608,848