System and method for image-based food item, search, design, and culinary fulfillment

1. A system for image-based food item search, design and culinary fulfillment, comprising:

a computing device comprising a memory, a processor, and a non-volatile data storage device;

a recipe database stored on the non-volatile data storage device, the recipe database comprising a plurality of recipes, each recipe comprising a food type, a first list of required ingredients and a first required culinary skill;

a patron profile database stored on the non-volatile data storage device, the patron profile database comprising a plurality of patron profiles, each patron profile comprising:

a patron preference; and

a patron review for one or more target food item recommendations, each target food item recommendation comprising a second list of required ingredients and a second required culinary skill;

a first machine learning algorithm configured to identify associations among the patron preferences, the first lists of required ingredients, and the first required culinary skills;

a food image engine comprising a first plurality of programming instructions stored in the memory which, when operating on the processor, causes the computing device to:

obtain a food item image;

process the food item image to extract image features;

input the extracted image features into a second machine learning algorithm configured to recognize and output a target food item based on the image features; and

send the output target food item to a prediction engine; and

a prediction engine comprising a second plurality of programming instructions stored in the memory which, when operating on the processor, causes the computing device to:

receive the target food item;

identify and extract a set of features associated with the target food item;

input the identified set of features associated with the target food item into a third machine learning algorithm configured to predict a match from a hypothesis space of the set of source ingredients to the received target food item;

output a predicted ingredient list comprising a set of source ingredients and their proportions; and

send the predicted ingredient list to a food item design engine; and

a food item design engine comprising a third plurality of programming instructions stored in the memory which, when operating on the processor, causes the computing device to:

convert the patron preferences, recipes, food items, and patron reviews to a first set of vector representations;

pass the vector representations through the first machine learning algorithm to identify associations among the patron preferences, the first lists of required ingredients, and the first required culinary skills;

receive a predicted ingredient list from the prediction engine;

convert the predicted ingredient list to a second set of vector representations;

pass the second set of vector representations through a fourth machine learning algorithm to obtain a best fit between the predicted ingredient list and the identified associations, the best fit comprising a third required list of ingredients and a third culinary skill; and

generate a target food item recommendation from the best fit.

2. The system of claim 1 , wherein the patron profile is updated with a patron review of the target food item recommendation received from the user.

3. The system of claim 1 , wherein the patron preference is based on social media information retrieved from a social media network.

4. The system of claim 1 , wherein the patron preference is based on nutritional data retrieved from a third-party resource over a network.

5. The system of claim 4 , wherein the nutritional data comprises allergy information.

6. The system of claim 1 , wherein the second machine learning algorithm is used to construct a food item model; and

wherein the food item model is configured to recognize and output a target food item based on the image features.

7. The system of claim 1 , wherein the third machine learning algorithm is used to construct a prediction model; and

wherein the prediction model is configured to predict a match from the hypothesis space of the set of source ingredients to the received target food item.

8. A method for image-based food item search, design and culinary fulfillment, comprising the steps of:

storing a recipe database on a non-volatile data storage device of a computing device comprising a memory, a processor, and the non-volatile data storage device, the recipe database comprising a plurality of recipes, each recipe comprising a food type, a first list of required ingredients and a first required culinary skill;

storing a patron profile database stored on the non-volatile data storage device, the patron profile database comprising a plurality of patron profiles, each patron profile comprising:

a patron preference; and

a patron review for one or more food item recommendations, each food item recommendation comprising a second list of required ingredients and a second required culinary skill;

configuring a first machine learning algorithm to identify associations among the patron preferences, the first lists of required ingredients, and the first required culinary skills;

using a food image engine operating on the computing device to:

obtaining a food item image;

processing the food item image to extract image features;

inputting the extracted image features into a second machine learning algorithm configured to recognize and output a target food item based on the image features;

sending the output target food item to a prediction engine;

using a prediction engine operating on the computing device to:

receiving the target food item;

identifying and extract a set of features associated with the target food item;

inputting the identified set of features associated with the target food item into a third machine learning algorithm configured to predict a match from a hypothesis space of the set of source ingredients to the received target food item;

outputting a predicted ingredient list comprising a set of source ingredients and their proportions;

sending the predicted ingredient list to a food item design engine;

using a prediction engine operating on the computing device to:

converting the patron preferences, recipes, food items, and patron reviews to a first set of vector representations;

passing the vector representations through the first machine learning algorithm to identify associations among the patron preferences, the first lists of required ingredients, and the first required culinary skills;

receiving a predicted ingredient list from the prediction engine;

converting the predicted ingredient list to a second set of vector representations;

passing the second set of vector representations through a fourth machine learning algorithm to obtain a best fit between the predicted ingredient list and the identified associations, the best fit comprising a third required list of ingredients and a third culinary skill; and

generating a target food item recommendation from the best fit.

9. The method of claim 8 , wherein the patron profile is updated with a patron review of the food item recommendation received from the user.

10. The method of claim 8 , wherein the patron preference is based on social media information retrieved from a social media network.

11. The method of claim 8 , wherein the patron preference is based on nutritional data retrieved from a third-party resource over a network.

12. The method of claim 11 , wherein the nutritional data comprises allergy information.

13. The method of claim 8 , wherein the second machine learning algorithm is used to construct a food item model; and

wherein the food item model is configured to recognize and output a target food item based on the image features.

14. The method of claim 8 , wherein the third machine learning algorithm is used to construct a prediction model; and

wherein the prediction model is configured to predict a match from the hypothesis space of the set of source ingredients to the received target food item.