IP Library Granted Patent US 11,069,045
Granted Patent B2
US 11,069,045 · App. 16/054,398 · Granted Jul 20, 2021

Food preparation assistance using image processing

Inventors: Jeremy R. Fox (Georgetown, TX); Christian Compton (Austin, TX); Lisa Seacat DeLuca (Baltimore, MD); Trudy L. Hewitt (Cary, NC)
Assignee: International Business Machines Corporation
G06T7/0002A23L5/00G06F16/951G06K9/00711G06N20/00G08B21/0202G08B21/0476G06K2009/00738G06T2207/30128
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,069,045
App. No.
16/054,398
Granted
Jul 20, 2021
Kind
B2
Abstract

A computer system monitors food preparation. Images of food preparation by a user are captured via one or more image capture devices disposed within an area containing food preparation items. A food preparation process being performed by the user is determined. Image processing is performed on the captured images to monitor the food preparation process and detect an event. The user is notified of the detected event, and provided information pertaining to the event. Embodiments of the present invention further include a method and program product for monitoring food preparation in substantially the same manner described above.

Claims (54)

1. A computer-implemented method of monitoring food preparation comprising:

capturing images of food preparation by a user via one or more image capture devices disposed within an area containing food preparation items;

determining, via a processor, a food preparation process being performed by the user, wherein determining the food preparation process comprises identifying a plurality of steps for preparing a food item according to the food preparation process, the plurality of steps including a seasoning step;

performing, via the processor, image processing on the captured images to monitor the food preparation process and detect an event;

notifying the user of the detected event, via the processor, and provide information pertaining to the event;

receiving feedback from the user pertaining to food preparation; and

utilizing a trained machine learning model to learn user preferences and adjust recommendations to the user to perform a step of the plurality of steps for preparing the food item according to the learned user preferences, wherein adjusting the recommendations includes recommending using a different amount of an ingredient from the amount indicated in a step of the plurality of steps for preparing the food item.

2. The computer-implemented method of claim 1 , wherein determining a food preparation process further comprises:

performing image processing on the captured images to determine the food preparation process being performed.

3. The computer-implemented method of claim 1 , wherein the event pertains to a safety issue, and the information pertaining to the event includes one or more operations to resolve the safety issue.

4. The computer-implemented method of claim 1 , further comprising:

aggregating data pertaining to the food preparation process performed by the user; and

comparing the aggregated data to prior food preparation processes to determine feedback for the user.

5. The computer-implemented method of claim 1 , further comprising:

receiving data from a population of users and classifying characteristics of food preparation items within the data to correlate classifications to resulting prepared food; and

adjusting recommendations to the user to prepare food based on the correlations and corresponding environment of the user.

6. The computer-implemented method of claim 1 , further comprising:

enabling locks of one or more from a group of storage items and appliances within the area in response to occurrence of predetermined conditions.

7. A computer system for monitoring food preparation, the computer system comprising:

one or more computer processors;

one or more computer readable storage media;

program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, the program instructions comprising instructions to:

capture images of food preparation by a user via one or more image capture devices disposed within an area containing food preparation items;

determine a food preparation process being performed by the user, wherein determining the food preparation process comprises identifying a plurality of steps for preparing a food item according to the food preparation process, the plurality of steps including a seasoning step;

perform image processing on the captured images to monitor the food preparation process and detect an event;

notify the user of the detected event, and provide information pertaining to the event;

receive feedback from the user pertaining to food preparation; and

utilize a trained machine learning model to learn user preferences and adjust recommendations to the user to perform a step of the plurality of steps for preparing the food item according to the learned user preferences, wherein adjusting the recommendations includes recommending using a different amount of an ingredient from the amount indicated in a step of the plurality of steps for preparing the food item.

8. The computer system of claim 7 , wherein the instructions to determine a food preparation process further comprise instructions to:

perform image processing on the captured images to determine the food preparation process being performed.

9. The computer system of claim 7 , wherein the event pertains to a safety issue, and the information pertaining to the event includes one or more operations to resolve the safety issue.

10. The computer system of claim 7 , further comprising instructions to:

aggregate data pertaining to the food preparation process performed by the user; and

compare the aggregated data to prior food preparation processes to determine feedback for the user.

11. The computer system of claim 7 , further comprising instructions to:

receive data from a population of users and classifying characteristics of food preparation items within the data to correlate classifications to resulting prepared food; and

adjust recommendations to the user to prepare food based on the correlations and corresponding environment of the user.

12. The computer system of claim 7 , further comprising instructions to:

enable locks of one or more from a group of storage items and appliances within the area in response to occurrence of predetermined conditions.

13. A computer program product for monitoring food preparation, the computer program product comprising one or more computer readable storage media collectively having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:

capture images of food preparation by a user via one or more image capture devices disposed within an area containing food preparation items;

determine a food preparation process being performed by the user, wherein determining the food preparation process comprises identifying a plurality of steps for preparing a food item according to the food preparation process, the plurality of steps including a seasoning step;

perform image processing on the captured images to monitor the food preparation process and detect an event;

notify the user of the detected event, and provide information pertaining to the event;

receive feedback from the user pertaining to food preparation; and

utilize a trained machine learning model to learn user preferences and adjust recommendations to the user to perform a step of the plurality of steps for preparing the food item according to the learned user preferences, wherein adjusting the recommendations includes recommending using a different amount of an ingredient from the amount indicated in a step of the plurality of steps for preparing the food item.

14. The computer program product of claim 13 , wherein the instructions to determine a food preparation process further comprise instructions to:

perform image processing on the captured images to determine the food preparation process being performed.

15. The computer program product of claim 13 , wherein the event pertains to a safety issue, and the information pertaining to the event includes one or more operations to resolve the safety issue.

16. The computer program product of claim 13 , further comprising instructions to:

aggregate data pertaining to the food preparation process performed by the user; and

compare the aggregated data to prior food preparation processes to determine feedback for the user.

17. The computer program product of claim 13 , further comprising instructions to:

enable locks of one or more from a group of storage items and appliances within the area in response to occurrence of predetermined conditions.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 3, 2018
From: FOX, JEREMY R.; COMPTON, CHRISTIAN; DELUCA, LISA SEACAT; HEWITT, TRUDY L.
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 046551/0942 →
Continuity (1)
Related Publication 20200043156A1 · Feb 6, 2020