Method and system for foodstuff identification
A method for foodstuff identification can include: detecting a trigger event; sampling a measurement set; optionally determining candidate measurements for subsequent analysis based on the measurement set; optionally determining a set of food parameter values from the measurements; optionally selecting a food parameter value for use; determining a cooking instruction based on the food parameter value; automatically operating the appliance based on the cooking instructions; optionally determining a foodstuff trajectory relative to the cook cavity; optionally training one or more modules; and/or any other suitable elements.
1. An appliance comprising:
a cooking cavity;
a camera system mounted above the cooking cavity; and
a processing system configured to:
receive, from the camera system, a series of images of a foodstuff;
select a subset of images, from the series of images, based on a confidence score associated with each image of the subset of images satisfying a threshold, wherein the confidence score is based on a visibility of the foodstuff in each image;
determine, based on the subset of images, one or more food parameter values associated with the foodstuff;
determine, based on a determination of the one or more food parameter values, one or more cooking instructions; and
operate, based on a determination of the one or more cooking instructions, the appliance.
2. The appliance of claim 1 , wherein the processing system is further configured to:
detect a triggering event; and
sample a series of measurements from the camera system, wherein measurements within the series of measurements are determined after the triggering event.
3. The appliance of claim 2 , wherein the processing system is further configured to:
determine, using the series of measurements, a foodstuff trajectory relative to the cooking cavity; and
adjusting, based on the foodstuff trajectory, a first cooking instruction of the one or more cooking instructions.
4. The appliance of claim 1 , further comprising one or more of:
a sensor configured to detect an opening or closing of a door; or
a sensor configured to detect one or more light changes.
5. The appliance of claim 1 , wherein the processing system is further configured to
determine the one or more food parameter values associated with the foodstuff using a neural network.
6. The appliance of claim 1 , wherein the processing system is further configured to
receive, from the camera system, a second series of images of a second foodstuff;
select a second subset of images, from the second series of images, based on a confidence score associated with each image of the second subset of images satisfying the threshold;
determine, based on the second subset of images, one or more second food parameter values associated with the second foodstuff;
determine, based on a determination of the one or more second food parameter values, one or more second cooking instructions; and
operate, based on a determination of the one or more second cooking instructions, the appliance.
7. The appliance of claim 6 , wherein the processing system is further configured to:
detect that the foodstuff has been removed from the cooking cavity;
adjust, based on removal of the foodstuff, a first cooking instruction of the one or more second cooking instructions; and
operate, based on the one or more adjusted second cooking instructions, the appliance to cook the second foodstuff.
8. The appliance of claim 1 , wherein the processing system is further configured to:
send one or more notifications to a user device, wherein the one or more notifications comprises a status of the foodstuff.
9. The appliance of claim 1 , wherein the camera system comprises a stereo camera.
10. The appliance of claim 1 , wherein the camera system is mounted to a top surface of the appliance and is angled outward relative to the cook cavity.
11. A method comprising:
receiving, from a camera system mounted above a cooking cavity of an appliance, a series of images of a foodstuff;
selecting a subset of images, from the series of images, based on a confidence score associated with each image of the subset of images satisfying a threshold, wherein the confidence score is based on a visibility of the foodstuff in each image;
determining, based on the subset of images, one or more food parameter values associated with the foodstuff;
determining, based on a determination of the one or more food parameter values, one or more cooking instructions; and
operating, based on a determination of the one or more cooking instructions, the appliance.
12. The method of claim 11 , further comprising:
detecting a triggering event; and
sampling a series of measurements from the camera system, wherein measurements within the series of measurements are determined after the triggering event.
13. The method of claim 12 , further comprising:
determine, using the series of measurements, a foodstuff trajectory relative to the cooking cavity; and
adjusting, based on the foodstuff trajectory, a first cooking instruction of the one or more cooking instructions.
14. The method of claim 11 , wherein the one or more food parameter values are determined based on or more of:
detecting an opening or closing of a door; or
detecting one or more light changes.
15. The method of claim 11 , further comprising:
determining, using a neural network, the one or more food parameter values associated with the foodstuff.
16. The method of claim 11 , further comprising:
receiving, from the camera system, a second series of images of a second foodstuff;
selecting a second subset of images, from the second series of images, based on a confidence score associated with each image of the second subset of images satisfying the threshold;
determining, based on the second subset of images, one or more second food parameter values associated with the second foodstuff;
determining, based on a determination of the one or more second food parameter values, one or more second cooking instructions; and
operating, based on a determination of the one or more second cooking instructions, the appliance.
17. The method of claim 16 , further comprising:
detecting that the foodstuff has been removed from the cooking cavity;
adjusting, based on removal of the foodstuff, a first cooking instruction of the one or more second cooking instructions; and
operating, based on the one or more adjusted second cooking instructions, the appliance to cook the second foodstuff.
18. The method of claim 11 , further comprising:
sending one or more notifications to a user device, wherein the one or more notifications comprises a status of the foodstuff.
19. The method of claim 11 , wherein the camera system comprises a stereo camera.
20. The method of claim 11 , wherein the camera system is mounted to a top surface of the appliance and is angled outward relative to the cook cavity.