Method and system for foodstuff identification
A method for foodstuff identification can include: detecting a trigger event; sampling a measurement set; optionally determining candidate measurements for subsequent analysis based on the measurement set; optionally determining a set of food parameter values from the measurements; optionally selecting a food parameter value for use; determining a cooking instruction based on the food parameter value; automatically operating the appliance based on the cooking instructions; optionally determining a foodstuff trajectory relative to the cook cavity; optionally training one or more modules; and/or any other suitable elements.
1. A system, comprising:
an appliance comprising a cook cavity and a camera system mounted above the cook cavity; and
a processing system, configured to:
detect a start trigger event at the appliance;
receive a series of measurements from the camera system, wherein measurements within the series of measurements are determined after the start trigger;
for each measurement in the series of measurements, determine a foodstuff score associated with a visibility of a foodstuff within the respective measurement, thereby producing a set of foodstuff scores;
select a candidate measurement set from the series of measurements based on the set of foodstuff scores;
calculate a food parameter value using the candidate measurement set;
determine a foodstuff translation trajectory relative to the cook cavity using the series of measurements;
determine a cooking instruction based on the foodstuff translation trajectory and the food parameter value; and
automatically operate the appliance based on the cooking instruction.
2. The system of claim 1 , wherein the series of measurements comprises a series of images sampled by the camera system.
3. The system of claim 2 , wherein the camera system comprises a stereo camera.
4. The system of claim 2 , wherein the camera system is mounted to a top surface of the appliance, and is angled outward relative to the cook cavity.
5. The system of claim 1 , wherein the food parameter value is a foodstuff identification determined using a trained neural network.
6. The system of claim 1 , wherein:
the series of measurements comprises a series of images;
determining the foodstuff score comprises calculating a proportion of the respective measurement encompassed by the foodstuff; and
selecting the candidate measurement set comprises selecting a subset of images from the series of images based on the foodstuff score exceeding a predetermined proportion of each images of the subset.
7. The system of claim 1 , wherein selecting the candidate measurement set comprises using motion detection to identify a measurement of the series of measurements that depicts a foodstuff within a region of interest, and when the measurement is identified, using the measurement to calculate the food parameter value.
8. The system of claim 1 , wherein the food parameter value is determined based on M of N agreements of food parameter values.
9. The system of claim 1 , wherein the processing system is additionally configured to, for an image in the series of measurements, determine an optical flow field based on the image and a prior image in the series of measurements, and wherein determining a foodstuff translation trajectory relative to the cook cavity using the series of measurements comprises using the optical flow field.