Ultraviolet light and machine learning-based assessment of food item quality
The disclosed technology provides for determining infection in food items using image data of the food items under ultra-violet (UV) light. A method includes performing object detection on the image data to identify a bounding box around each of the food items in the image data, determining, for each food item, an infection presence metric by applying a model to the bounding box, the model being trained using image training data of other food items under UV light, the image training data being annotated based on previous identifications of a first portion of the other food items having infection features and a second portion having healthy quality features, and determining, based on a determination that the infection presence metric for each of the food items indicates presence of an infection, an infection coverage metric for the food item.
1 . A method for determining infection presence in food items using image data, the method comprising:
receiving, by a computing system and from an imaging device, image data of food items under UV light;
determining, by the computing system, an infection presence metric for each of the food items by applying a model to the image data, wherein applying the model to the image data comprises:
applying a grid structure to the image data to assign each food item in the image data a respective grid index that identifies a placement of the food item in the grid structure;
wherein the model was trained using image training data of other food items under UV light, wherein the image training data comprises a grid of other food items, each other food item assigned a respective grid index, the image training data being annotated based on previous identifications of a first portion of the other food items as having infection features and a second portion of the other food items as having healthy quality features, wherein the other food items are of a same food type as the food items;
identifying, by the computing system and using the respective grid indices, an infected group of the food items in the image data as having respective infection presence metrics that satisfy a threshold infection level, the threshold infection level indicating that the food items are infected based on the first portion and the second portion; and
returning, by the computing system, the infected group of the food items in the image data as having respective infection presence metrics and the infection presence metric for the food items in the image data.
2 . The method of claim 1 , further comprising:
determining, by the computing system and based on the infection presence metrics for the food items, an infection coverage metric for each of the food items; and
returning, by the computing system, the infection coverage metric for the food items in the image data.
3 . The method of claim 2 , wherein the infection coverage metric for each food item is a percentage of a surface of the food item that includes features indicative of the infection.
4 . The method of claim 1 , wherein returning, by the computing system, the infection presence metric for the food items in the image data comprises transmitting the infection presence metric to a user device for presentation in a GUI display at the user device.
5 . The method of claim 1 , wherein the infection presence metric for each food item is a string value indicating a healthy food item or an infected food item.
6 . The method of claim 1 , further comprising:
determining, by the computing system, an edibility metric for the group of food items by applying an edibility model to the image data, wherein the edibility model was trained using training time series image data of infected food items, the training time series image data being annotated with previous identifications of infection surface coverage correlating to lengths of time of edibility of the infected food items, wherein the infected food items are of the same food type as the food items; and
returning, by the computing system, the edibility metric for the group of food items.
7 . The method of claim 6 , wherein determining, by the computing system, the edibility metric for the group of food items comprises predicting a length of time of edibility for the group of food items.
8 . The method of claim 1 , further comprising
retrieving, by the computing system, from a data store, and for each of the food items, the infection presence metric for the food item;
identifying, by the computing system, supply chain information for the food item that includes a preexisting supply chain schedule and destination for the food item;
determining, by the computing system, whether to modify the supply chain information for the food item based on the infection presence metric;
in response to a determination to modify the supply chain information, generating, by the computing system, modified supply chain information based on the infection presence metric, wherein the modified supply chain information includes one or more of a modified supply chain schedule and modified destination for the food item; and
transmitting, by the computing system, the modified supply chain information to one or more supply chain actors to implement the modified supply chain information.
9 . The method of claim 8 , wherein determining, by the computing system, whether to modify the supply chain information for the food item based on the infection presence metric comprises determining that the infection presence metric satisfies the threshold infection level.
10 . A system for determining infection presence in food items using image data, the system comprising:
at least one light source configured to illuminate food items of a same food type, wherein the at least one light source emits ultra-violet (UV) light;
one or more imaging devices configured to capture image data for the food items when the food items are illuminated by the at least one light source; and
at least one computing system in communication with the one or more imaging devices, the at least one computing system configured to:
receive, by a computing system and from an imaging device, image data of food items under UV light;
determine, by the computing system, an infection presence metric for each of the food items by applying a model to the image data, wherein applying the model to the image data comprises:
applying a grid structure to the image data to assign each food item in the image data a respective grid index that identifies a placement of the food item in the grid structure;
wherein the model was trained using image training data of other food items under UV light, wherein the image training data comprises a grid of other food items, each other food item assigned a respective grid index, the image training data being annotated based on previous identifications of a first portion of the other food items as having infection features and a second portion of the other food items as having healthy quality features, wherein the other food items are of a same food type as the food items;
identify, by the computing system and using the respective grid indices, an infected group of the food items in the image data as having respective infection presence metrics that satisfy a threshold infection level, the threshold infection level indicating that the food items are infected based on the first portion and the second portion; and
return, by the computing system, the infected group of the food items in the image data as having respective infection presence metrics and the infection presence metric for the food items in the image data.
11 . The system of claim 10 , wherein the at least one computing system is further configured to:
determine, by the computing system and based on the infection presence metrics for the food items, an infection coverage metric for each of the food items; and
return, by the computing system, the infection coverage metric for the food items in the image data.
12 . The system of claim 11 , wherein the infection coverage metric for each food item is a percentage of a surface of the food item that includes features indicative of the infection.
13 . The system of claim 10 , wherein returning, by the computing system, the infection presence metric for the food items in the image data comprises transmitting the infection presence metric to a user device for presentation in a GUI display at the user device.
14 . The system of claim 10 , wherein the infection presence metric for each food item is a string value indicating a healthy food item or an infected food item.
15 . The system of claim 10 , wherein the at least one computing system is configured to:
determine, by the computing system, an edibility metric for the group of food items by applying an edibility model to the image data, wherein the edibility model was trained using training time series image data of infected food items, the training time series image data being annotated with previous identifications of infection surface coverage correlating to lengths of time of edibility of the infected food items, wherein the infected food items are of the same food type as the food items; and
return, by the computing system, the edibility metric for the group of food items.
16 . A non-transitory computer-readable medium encoding instructions operable to cause data processing apparatus to perform operations comprising:
receiving, by a computing system and from an imaging device, image data of food items under UV light;
determining, by the computing system, an infection presence metric for each of the food items by applying a model to the image data, wherein applying the model to the image data comprises:
applying a grid structure to the image data to assign each food item in the image data a respective grid index that identifies a placement of the food item in the grid structure;
wherein the model was trained using image training data of other food items under UV light, wherein the image training data comprises a grid of other food items, each other food item assigned a respective grid index, the image training data being annotated based on previous identifications of a first portion of the other food items as having infection features and a second portion of the other food items as having healthy quality features, wherein the other food items are of a same food type as the food items;
identifying, by the computing system and using the respective grid indices, an infected group of the food items in the image data as having respective infection presence metrics that satisfy a threshold infection level, the threshold infection level indicating that the food items are infected based on the first portion and the second portion; and
returning, by the computing system, the infected group of the food items in the image data as having respective infection presence metrics and the infection presence metric for the food items in the image data.
17 . The non-transitory computer-readable medium of claim 16 , further comprising instructions operable to cause data processing apparatus to perform operations comprising:
determining, by the computing system and based on the infection presence metrics for the food items, an infection coverage metric for each of the food items; and
returning, by the computing system, the infection coverage metric for the food items in the image data.
18 . The non-transitory computer-readable medium of claim 17 , wherein the infection coverage metric for each food item is a percentage of a surface of the food item that includes features indicative of the infection.
19 . The non-transitory computer-readable medium of claim 16 , wherein returning, by the computing system, the infection presence metric for the food items in the image data comprises transmitting the infection presence metric to a user device for presentation in a GUI display at the user device.
20 . The non-transitory computer-readable medium of claim 16 , wherein the infection presence metric for each food item is a string value indicating a healthy food item or an infected food item.