Multi-task machine learning model for event detection
A computer-implemented method for machine learning model operation can include, by one or more processors executing program instructions: providing a first training dataset comprising a plurality of images and associated object detection labels, providing a second training dataset comprising a plurality of images and associated classification labels, and providing a machine learning model comprising a model backbone, an object detection task head, and a classification task head. The method can further include training the machine learning model by training the object detection task head using the first training dataset and training the classification task head using the second training dataset. The method can further include deploying the trained machine learning model that includes the trained model backbone and the trained object detection task head but does not include the trained classification task head.
1. A computer-implemented method for machine learning model operation, the computer-implemented method comprising, by one or more processors executing program instructions:
providing a first training dataset comprising a plurality of images and associated object detection labels;
providing a second training dataset comprising a plurality of images and associated classification labels, wherein the second training dataset does not include object detection labels;
providing a machine learning model comprising a model backbone, an object detection task head, and a classification task head;
training the machine learning model, wherein training the machine learning model comprises:
training the object detection task head using the first training dataset; and
training the classification task head using the second training dataset; and
deploying the trained machine learning model for use, wherein the trained machine learning model includes the trained model backbone and the trained object detection task head, and wherein the trained machine learning model does not include the trained classification task head.
2. The computer-implemented method of claim 1 , wherein the training of the model backbone is improved for purposes of object detection by incorporation of the training of the classification task head in the training of the machine learning model.
3. The computer-implemented method of claim 1 , wherein the machine learning model comprises a multi-task learning model.
4. The computer-implemented method of claim 1 further comprising, by the one or more processors executing program instructions:
executing the trained machine learning model on a new image using the trained model backbone and the trained object detection task head to generate, for the new image, at least: an object detection label and a confidence score.
5. The computer-implemented method of claim 4 , wherein executing the trained machine learning model further generates: a head pose label.
6. The computer-implemented method of claim 1 , wherein each of the object detection labels comprises a bounding box identifying a position of an object in an image.
7. The computer-implemented method of claim 1 , wherein each of the classification labels comprises a yes or no indication related to a presence of an object in an image.
8. The computer-implemented method of claim 6 , wherein the object comprises at least one of: a hand touching a mobile device, a torso not wearing a seatbelt, a face looking inside a vehicle, a face, or a hand touching a food or drink item.
9. The computer-implemented method of claim 1 , wherein the machine learning model comprises three or more task heads.
10. The computer-implemented method of claim 1 , wherein classification labels of images of the second training dataset are received from drivers of vehicles.
11. The computer-implemented method of claim 1 , wherein the trained machine learning model is deployed in a dash cam of a vehicle.
12. The computer-implemented method of claim 1 , wherein training the machine learning model further comprises:
training the model backbone using both the first training dataset and the second training dataset.
13. A system comprising:
one or more computer-readable storage mediums having program instructions embodied therewith; and
one or more processors configured to execute the program instructions to cause the system to perform the computer-implemented method of claim 1 .
14. The system of claim 13 , wherein the training of the model backbone is improved for purposes of object detection by incorporation of the training of the classification task head in the training of the machine learning model.
15. The system of claim 13 , wherein the machine learning model comprises a multi-task learning model.
16. The system of claim 13 , wherein the program instructions are executable by the one or more processors to cause the system to perform further operations comprising:
executing the trained machine learning model on a new image using the trained model backbone and the trained object detection task head to generate, for the new image, at least: an object detection label and a confidence score.
17. The system of claim 16 , wherein executing the trained machine learning model further generates: a head pose label.
18. The system of claim 13 , wherein each of the object detection labels comprises a bounding box identifying a position of an object in an image.
19. The system of claim 13 , wherein each of the classification labels comprises a yes or no indication related to a presence of an object in an image.
20. A computer program product comprising one or more computer-readable storage mediums having program instructions embodied therewith, the program instructions executable by one or more processors to cause the one or more processors to perform the computer-implemented method of claim 1 .