System, method, and software program product for a self-shopping refrigerator
The invention provides a refrigerator system, a method and a software program product for controlling and/or advising a diet for a user. The refrigerator system has a refrigerator including a light and a camera that captures an image of contents of the refrigerator. The image is processed by a graphics processing unit (GPU) integrated with the refrigerator and/or a cloud server. The GPU analyses the image to recognise a quantity, a quality, and/or a kind of food articles and/or food packages such as, vegetables, fruits, beverages, etc. A central processing unit (CPU)/GPU of the refrigerator provides diet plans, recipe recommendations, identifies grocery replenishment requirements, updates and shares shopping lists with food delivery agents, etc., based on the analysis of the refrigerator contents. The invention helps in minimizing food wastage, expanding meal options considering expiry dates, and maximizing benefits of a healthy diet by recommending and controlling the user's diet.
1 . A refrigerator system comprising:
a camera is configured to capture a photograph of naked unpackaged food;
a GPU processing unit configured to analyze the photograph using artificial intelligence for determining naked unpackaged food quantity, food quality, or type of food and produce alphanumeric values describing naked unpackaged food quantity, food quality, or type of food into computer readable memory, and the derived information concerning the naked unpackaged food quantity, quality and/or the type of food is configured to be used in controlling and/or advising on the diet of a human and/or mammal, and
the derived information concerning the naked unpackaged food quantity, quality and/or the type of food is configured to be used in identifying deficiencies in the contents of the refrigerator ( 225 , 226 , 227 , or 228 ).
2 . A refrigerator system ( 200 ), as claimed in claim 1 , characterized in that, and the identified deficiencies are configured to be used in updating a shopping list and/or the shopping list is configured to be sent to a grocery shop and/or food delivery agent ( 700 ).
3 . A refrigerator system ( 200 ), as claimed in claim 1 , characterized in that, the derived information concerning the food quantity, quality and/or the type of food is configured to be used in identifying at least one recipe(s) for a meal that could be implemented with the food ( 259 ).
4 . A refrigerator system ( 200 ), as claimed in claim 3 , characterized in that, the derived information concerning the food quantity, quality and/or the type of food is configured to be used in identifying a series of recipes for meals that could be implemented with the food ( 259 ) based upon one or more of the following criteria:
minimisation of waste of food ( 259 ),
maximisation of diet variety as a function of time,
maximisation of the healthiness of diet.
5 . A refrigerator system ( 200 ), as claimed in claim 1 , characterized in that, visual search is configured to be conducted with the captured image or part of the captured image as a query term, and/or text in at least one food package is identified from the image, for example with OCR, and the identified text is used as a query term.
6 . A refrigerator system ( 200 ), as claimed in claim 5 , characterized in that, the best before date on the food package is identified, and the recognized best before date is configured to be used in compiling a list of recipes for meals as a function of time, which when implemented consumes the maximum amount of food ( 259 ) before the best before date on any of the food ( 259 ) has expired and/or the most expensive food(s) ( 259 ) has expired.
7 . A refrigerator system ( 200 ), as claimed in claim 1 , characterized in that, at least one surface inside the refrigerator ( 225 , 226 , 227 , or 228 ) has a scale ( 605 ) for weighing food ( 259 ) and/or a device ( 610 ) for moving food ( 259 ) mechanically inside the refrigerator ( 225 , 226 , 227 , or 228 ).
8 . A refrigerator system ( 200 ), as claimed in claim 1 , characterized in that, the refrigerator system ( 200 ) is configured to receive images of meals and/or food articles and/or food packages from the user and the graphical processing unit GPU ( 210 or 240 ) is configured to analyze the received image, and recognize the food quantity, quality and/or the type of food from the image, and optionally use the food quantity, quality and/or the type of food from the image to compute a changed recipe list, and optionally send the changed recipe list to user's mobile phone and/or user terminal device ( 215 ).
9 . A refrigerator system ( 200 ), as claimed in claim 1 , characterized in that, the refrigerator ( 225 , 226 , 227 , or 228 ) is configured to receive data files from a cloud server ( 205 ), and optionally the data files are deep learning data files configured to facilitate the recognition of a food article by the GPU ( 210 or 240 ) from an image of the food article, and/or
the at least one image and/or data derived from at least one image is configured to be communicated to a social network account.
10 . A method for controlling and/or advising on the diet of a human and/or a mammal, comprising at least one refrigerator ( 225 , 226 , 227 , or 228 ), at least one GPU processing unit ( 210 or 240 ), at least one camera ( 235 , 236 , 237 , or 238 ), characterized by the following steps,
capturing a photograph of naked unpackaged food using at least one camera; and
analyzing the photograph, using the GPU processing unit and artificial intelligence, for determining the naked unpackaged food quantity, food quality, or type of food,
and producing alphanumeric values describing naked unpackaged food quantity, food quality, or type of food into computer readable memory, and the derived information concerning the naked unpackaged food quantity, quality and/or the type of food is used in controlling and/or advising on the diet of a human and/or mammal, and the derived information concerning the naked unpackaged food quantity, quality and/or the type of food is used in identifying deficiencies in the contents of the refrigerator.
11 . A method as claimed in claim 10 , characterized in that the identified deficiencies are used in updating a shopping list and/or the shopping list is sent to a grocery shop and/or food delivery agent ( 700 ).
12 . A method as claimed in claim 10 , characterized in that, the derived information concerning the food quantity, quality and/or the type of food is used in identifying at least one recipe(s) for a meal that could be implemented with the food ( 259 ).
13 . A method as claimed in claim 12 , characterized in that, the derived information concerning the food quantity, quality and/or the type of food is used in identifying a series of recipes for meals that could be implemented with the food ( 259 ) based upon one or more of the following criteria:
minimisation of waste of food ( 259 ),
maximisation of diet variety as a function of time,
maximisation of the healthiness of diet.
14 . A method as claimed in claim 10 , characterized in that, visual search is conducted on the captured image with the captured image or part of the captured image as a query term, and/or text in at least one food package is identified from the image, for example with OCR, and the identified text is used as a query term.
15 . A method as claimed in claim 14 , characterized in that, the best before date on the food package is identified, and the recognized best before date is used in compiling a list of recipes for meals as a function of time, which when implemented consumes the maximum amount of food ( 259 ) before the best before date on any of the food ( 259 ) has expired and/or on the most expensive food ( 259 ) has expired.
16 . A method as claimed in claim 10 , characterized in that, at least one surface inside the refrigerator ( 225 , 226 , 227 , or 228 ) has a scale ( 605 ) for weighing food ( 259 ) and/or a device ( 610 ) for moving food ( 259 ) mechanically inside the refrigerator ( 225 , 226 , 227 , or 228 ).
17 . A method as claimed in claim 10 , characterized in that, the refrigerator system ( 200 ) receives images of meals and/or food articles and/or food packages from the user and the graphical processing unit GPU ( 210 or 240 ) analyzes the received image, and recognizes the food quantity, quality and/or the type of food from the image, and optionally uses the food quantity, quality and/or the type of food from the image to compute a changed recipe list, and optionally sends the changed recipe list to user's mobile phone and/or user terminal device ( 215 ).
18 . A method as claimed in claim 10 , characterized in that, the refrigerator ( 225 , 226 , 227 , or 228 ) receives data files from a cloud server ( 205 ), and optionally the data files are deep learning data files configured to facilitate the recognition of a food article by the GPU ( 210 or 240 ) from an image of the food article, and/or
the at least one image and/or data derived from at least one image is communicated to a social network account.
19 . A software program product stored in a memory medium for controlling a refrigerator system, the software program product comprising programmed instructions for:
capturing a photograph of naked unpackaged food using at least one camera; and
analyzing the photograph, using a graphical processing unit GPU and artificial intelligence, for determining the naked unpackaged food quantity, food quality, or type of food,
and producing alphanumeric values describing naked unpackaged food quantity, food quality, or type of food into computer readable memory, and
the derived information concerning the naked unpackaged food quantity, quality and/or the type of food is used in controlling and/or advising on the diet of a human and/or mammal, and the derived information concerning the naked unpackaged food quantity, quality and/or the type of food is used in identifying deficiencies in the contents of the refrigerator.
20 . A software program product as claimed in claim 19 , characterized in that the identified deficiencies are used in updating a shopping list and/or the shopping list is sent to a grocery shop and/or food delivery agent ( 700 ).
21 . A software program product as claimed in claim 19 , characterized in that, the derived information concerning the food quantity, quality and/or the type of food is used in identifying at least one recipe(s) for a meal that could be implemented with the food ( 259 ).
22 . A software program product as claimed in claim 21 , characterized in that, the derived information concerning the food quantity, quality and/or type of food is used in identifying a series of recipes for meals that could be implemented with the food ( 259 ) based upon one or more of the following criteria:
minimisation of waste of food ( 259 ),
maximisation of diet variety as a function of time,
maximisation of the healthiness of diet.
23 . A software program product as claimed in claim 19 , characterized in that, visual search is conducted on the captured image with the captured image or part of the captured image as a query term, and/or text in at least one food package is identified from the image, for example with OCR, and the identified text is used as a query term.
24 . A software program product as claimed in claim 23 , characterized in that, the best before date on the food package is identified, and the recognized best before date is used in compiling a list of recipes for meals as a function of time, which when implemented consumes the maximum amount of food ( 259 ) before the best before date on any of the food ( 259 ) has expired and/or on the most expensive food ( 259 ) has expired.
25 . A software program product as claimed in claim 19 , characterized in that, at least one surface inside the refrigerator ( 225 , 226 , 227 , or 228 ) has a scale ( 605 ) for weighing food ( 259 ) and/or a device ( 610 ) for moving food ( 259 ) mechanically inside the refrigerator ( 225 , 226 , 227 , or 228 ).
26 . A software program product as claimed in claim 19 , characterized in that, the software program product receives images of meals and/or food articles and/or food packages from the user and the graphical processing unit GPU ( 210 or 240 ) analyzes the received image, and recognizes the food quantity, quality and/or the type of food from the image, and optionally uses the food quantity, quality and/or the type of food from the image to compute a changed recipe list, and optionally sends the changed recipe list to user's mobile phone and/or user terminal device ( 215 ).
27 . A software program product as claimed in claim 19 , characterized in that, the software program product receives data files from a cloud server ( 205 ), and optionally the data files are deep learning data files configured to facilitate the recognition of a food article by the GPU ( 210 or 240 ) from an image of the food article, and/or
the at least one image and/or data derived from at least one image is configured to be communicated to a social network account.