IP Library Granted Patent US 11,599,741
Granted Patent B1
US 11,599,741 · App. 16/774,796 · Granted Mar 7, 2023

Generating data in a messaging system for a machine learning model

Inventors: Zehao Xue (Los Angeles, CA); Zhou Ren (Bellevue, WA)
Assignee: SNAP INC.
G06K9/6253G06K9/6256G06N20/00G06V20/68G06V2201/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,599,741
App. No.
16/774,796
Granted
Mar 7, 2023
Kind
B1
Abstract

Systems and methods are provided for analyzing, by a computing device, location data associated with a location of the computing device to determine that an image or video captured using a messaging application on the computing device is captured near a food-related venue or event, receiving input related to food associated with the food-related venue or event, sending the image or video and the input related to food associated with the food-related venue or event to a computing system to train a machine learning model for food detection, and updating the messaging application to comprise the trained machine learning model for food detection.

Claims (58)

1. A method comprising:

analyzing, by a computing device, location data associated with a location of the computing device to determine that an image or video captured using a messaging application on the computing device is captured near a food-related venue or event;

receiving, by the computing device, input related to food associated with the food-related venue or event;

sending, by the computing device, the image or video and the input related to food associated with the food-related venue or event to a computing system to train a machine learning model for food detection;

updating, by the computing device, the messaging application to comprise the trained machine learning model for food detection;

detecting capture of a second image or video using the messaging application; and

determining that the second image or video comprises food using the trained machine learning model for food detection.

2. The method of claim 1 , wherein determining that the second image or video comprises food using the trained machine learning model for food detection comprises:

inputting the second image or video into the machine learning model for food detection;

generating a prediction value for the second image or video;

determining whether the prediction value exceeds a predetermined threshold value; and

determining that the second image or video comprises food based on the prediction value exceeding the predetermined threshold value.

3. The method of claim 1 , wherein analyzing the location data associated with the location of the computing device to determine that the image or video is captured near the food-related venue or event comprises comparing the location data to map data to determine that the location of the computing device is near the food-related venue or event.

4. The method of claim 1 , wherein the input is received in response to presented interactive features and the interactive features comprise a media overlay, a lens, or a request for a review of the food-related venue or event.

5. The method of claim 1 , wherein the input is received in response to presented interactive features and the interactive features comprise a request for an image of food from the food-related venue or event, and the image or video sent to the computing system is the requested image.

6. The method of claim 1 , wherein the computing system receives the image or video and associated input from the computing device and stores the received image or video and associated input in a database with a plurality of captured images and videos and associated input from a plurality of computing devices.

7. The method of claim 1 , wherein the trained machine learning model for food detection is further trained on a plurality of captured images and videos and input sent by a plurality of computing devices.

8. The method of claim 1 , further comprising:

detecting capture of a third image or video using the messaging application;

determining that the third image or video is captured near a food-related venue or event;

determining that the third image or video does not comprise food, using the trained machine learning model for food detection; and

presenting interactive features to incentivize capture of an image of food associated with the food-related venue or event.

9. The method of claim 8 , further comprising:

in response to the presented interactive features, receiving a fourth image; and

sending the fourth image to the computing system to update the trained machine learning model for food detection.

10. A computing device comprising:

one or more hardware processors; and

a computer-readable medium coupled with the one or more hardware processors, the computer-readable medium comprising instructions stored thereon that are executable by the one or more hardware processors to cause the computing device to perform operations comprising:

analyzing location data associated with a location of the computing device to determine that an image or video captured using a messaging application on the computing device is captured near a food-related venue or event;

receiving input related to food associated with the food-related venue or event;

sending the image or video and the input related to food associated with the food-related venue or event to a computing system to train a machine learning model for food detection;

updating the messaging application to comprise the trained machine learning model for food detection;

detecting capture of a second image or video using the messaging application; and

determining that the second image or video comprises food using the trained machine learning model for food detection.

11. The computing device of claim 10 , wherein determining that the second image or video comprises food using the trained machine learning model for food detection comprises:

inputting the second image or video into the machine learning model for food detection;

generating a prediction value for the second image or video;

determining whether the prediction value exceeds a predetermined threshold value; and

determining that the second image or video comprises food based on the prediction value exceeding the predetermined threshold value.

12. The computing device of claim 10 , wherein analyzing the location data associated with the location of the computing device to determine that the image or video is captured near the food-related venue or event comprises comparing the location data to map data to determine that the location of the computing device is near the food-related venue or event.

13. The computing device of claim 10 , wherein the input is received in response to presented interactive features and the interactive features comprise a media overlay, a lens, or a request for a review of the food-related venue or event.

14. The computing device of claim 10 , wherein the input is received in response to presented interactive features and the interactive features comprise a request for an image of food from the food-related venue or event, and the image or video sent to the computing system is the requested image.

15. The computing device of claim 10 , wherein the trained machine learning model for food detection is further trained on a plurality of captured images and videos and input sent by a plurality of computing devices.

16. The computing device of claim 10 , further comprising:

detecting capture of a third image or video using the messaging application;

determining that the third image or video is captured near a food-related venue or event;

determining that the third image or video does not comprise food, using the trained machine learning model for food detection; and

presenting interactive features to incentivize capture of an image of food associated with the food-related venue or event.

17. The computing device of claim 16 , the operations further comprising:

in response to the presented interactive features, receiving a fourth image; and

sending the fourth image to the computing system to update the trained machine learning model for food detection.

18. A non-transitory computer-readable medium comprising instructions stored thereon that are executable by at least one processor to cause a computing device to perform operations comprising:

analyzing location data associated with a location of the computing device to determine that an image or video captured using a messaging application on the computing device is captured near a food-related venue or event;

receiving input related to food associated with the food-related venue or event;

sending the image or video and the input related to food associated with the food-related venue or event to a computing system to train a machine learning model for food detection;

updating the messaging application to comprise the trained machine learning model for food detection;

detecting capture of a second image or video using the messaging application; and

determining that the second image or video comprises food using the trained machine learning model for food detection.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 13, 2023
From: XUE, ZEHAO; REN, ZHOU
To: SNAP INC.
Reel/Frame 062368/0770 →
Continuity (1)
Continuation 15829572 · Dec 1, 2017
Cited By (2)
US 12,369,745 US 12,705,872