System and method for recipe to image associations
System and method for associating images to recipe records. In one embodiment, a method is provided which includes: receiving an uploaded photograph of a recipe image; and searching a database for an image which matches to the uploaded image. The database search may, in one embodiment, make use of a convolutional neural network (CNN)-based classifier. When one of the records in the database is identified as matching, providing a portion of the data associated to the record to the user device; receiving feedback from the user; and based on the feedback, providing a remaining portion of the data associated to the identified record to the user device for logging thereat; or enabling the user to create a new consumable item record.
1. A method for enabling efficient association of a desired multi-ingredient consumable item to one of a plurality of multi-ingredient consumable item records in a database, individual ones of said plurality of multi-ingredient consumable item records being associated to a plurality of data and at least one image, respectively, said method comprising:
receiving at a server apparatus a photograph uploaded from a user device, said photograph comprising a photograph taken via a camera function of said user device of an image provided in association with a multi-ingredient consumable item;
searching said database for an image which matches to said image of said uploaded photograph;
when one of said individual ones of said plurality of multi-ingredient consumable item records is identified as being associated to said image which matches said uploaded photograph:
providing a portion of said data associated to said identified one of said individual ones of said plurality of multi-ingredient consumable item records to said user device;
receiving feedback from a user of said user device indicating whether said identified one of said individual ones of said plurality of multi-ingredient consumable items comprises said desired multi-ingredient consumable item; and
when said feedback indicates that said one of said identified one of said individual ones of said plurality of multi-ingredient consumable item records corresponds to the desired multi-ingredient consumable item, providing a remaining portion of said data associated to said identified one of said individual ones of said plurality of multi-ingredient consumable item records to said user device for logging thereat; and
when none of said individual ones of said plurality of multi-ingredient consumable item records is identified as being associated to said image, or when said feedback indicates that said one of said identified one of said individual ones of said plurality of multi-ingredient consumable item records does not correspond to the desired multi-ingredient consumable item, enabling said user to create a new consumable item record.
2. The method of claim 1 , wherein said act of searching said database comprises using a convolutional neural network (CNN)-based classifier to efficiently identify said image which matches said uploaded photograph.
3. The method of claim 1 , wherein said portion of said data comprises a title of said identified one of said individual ones of said plurality of multi-ingredient consumable item records presented in a list of titles each having some threshold level of similarity to said uploaded photograph.
4. The method of claim 1 , wherein said remaining portion of said data comprises data relating to a number of servings, each ingredient in said multi-ingredient consumable item, and/or nutrition data.
5. The method of claim 1 , further comprising providing said new consumable item record is for storage at said database.
6. The method of claim 1 , further comprising providing said new consumable item record for logging at said user device.
7. The method of claim 1 , wherein said new consumable item record comprises said uploaded photograph and a plurality of data relating thereto.
8. A non-transitory, computer readable medium comprising a plurality of instructions which are configured to, when executed, cause a server device to:
receive a photograph uploaded from a user device, said photograph comprising a photograph taken via a camera function of said user device of an image provided in association with a multi-ingredient consumable item;
search a database comprising a plurality of multi-ingredient consumable item records, individual ones of said records comprising a plurality of data and at least one image, said search comprising a search for an image which matches to said image of said uploaded photograph;
identify one of said individual ones of said plurality of multi-ingredient consumable item records as being associated to said image which matches said image of said uploaded photograph;
provide a portion of said data associated to said identified one of said individual ones of said plurality of multi-ingredient consumable item records to said user device; and
receive feedback from a user of said user device indicating whether said identified one of said individual ones of said plurality of multi-ingredient consumable items comprises a desired multi-ingredient consumable item; and
when said feedback indicates that said one of said identified one of said individual ones of said plurality of multi-ingredient consumable item records does not correspond to the desired multi-ingredient consumable item, said plurality of instructions are further configured to cause said server device to enable said user to create a new consumable item record comprising said uploaded photograph and a plurality of data relating thereto, said new consumable item record being provided for storage at said database.
9. The computer readable medium of claim 8 , wherein when said feedback indicates that said one of said identified one of said individual ones of said plurality of multi-ingredient consumable item records corresponds to the desired multi-ingredient consumable item, said plurality of instructions are further configured to cause said server device to provide a remaining portion of said data associated to said identified one of said individual ones of said plurality of multi-ingredient consumable item records to said user device for logging thereat.
10. The computer readable medium of claim 9 , wherein said plurality of instructions are further configured to cause said server device to enable said user to edit said remaining portion of said multi-ingredient consumable item record to create a new record, said new consumable item record being provided for storage at said database.
11. The computer readable medium of claim 8 , wherein said search of said database comprises utilization of a convolutional neural network (CNN)-based classifier to efficiently identify said image which matches said uploaded photograph.
12. The computer readable medium of claim 11 , wherein said CNN-based classifier comprises a modified GoogLeNet CNN, said modification comprising replacement of its softmax with a smaller softmax and utilization of said plurality of multi-ingredient consumable item records in said database.
13. A network apparatus configured to enable efficient association of a desired multi-ingredient consumable item to one of a plurality of multi-ingredient consumable item records in a database, individual ones of said plurality of multi-ingredient consumable item records being associated to a plurality of data and at least one image, respectively, said apparatus comprising:
one or more transceivers;
a storage apparatus; and
a processor configured to execute at least one computer application thereon, said computer application comprising a plurality of instructions which are configured to, when executed, cause said network apparatus to:
receive a photograph uploaded from a user device, said photograph comprising a photograph taken via a camera function of said user device of an image provided in association with a multi-ingredient consumable item;
search said database for an image which matches to said image of said uploaded photograph;
when one of said individual ones of said plurality of multi-ingredient consumable item records is identified as being associated to said image which matches said image of said uploaded photograph:
provide a portion of said data associated to said identified one of said individual ones of said plurality of multi-ingredient consumable item records to said user device; and
receive feedback from a user of said user device indicating whether said identified one of said individual ones of said plurality of multi-ingredient consumable items comprises said desired multi-ingredient consumable item.
14. The network apparatus of claim 13 , wherein when said feedback indicates that said one of said identified one of said individual ones of said plurality of multi-ingredient consumable item records corresponds to the desired multi-ingredient consumable item, providing a remaining portion of said data associated to said identified one of said individual ones of said plurality of multi-ingredient consumable item records to said user device for logging thereat.
15. The network apparatus of claim 13 , wherein when said feedback indicates that said one of said identified one of said individual ones of said plurality of multi-ingredient consumable item records does not correspond to the desired multi-ingredient consumable item, enabling said user to create a new consumable item record comprising said uploaded photograph and a plurality of data relating thereto, said new consumable item record being provided for storage at said database.
16. The network apparatus of claim 13 , wherein said search of said database comprises a utilization of a convolutional neural network (CNN)-based classifier to efficiently identify said image which matches said uploaded photograph.
17. The network apparatus of claim 16 , wherein said CNN-based classifier comprises a modified GoogLeNet CNN, said modification comprising replacement of its softmax with a smaller softmax and utilization of said plurality of multi-ingredient consumable item records in said database.
18. The network apparatus of claim 13 , wherein said portion of said data comprises a title of said identified one of said individual ones of said plurality of multi-ingredient consumable item records presented in a list of titles each having some threshold level of similarity to said uploaded photograph.