Intelligent projection point prediction for overhead objects
Methods and systems provide for the intelligent prediction of projection points of overhead objects in a workplace or similar environment. In one embodiment, the system receives one or more two-dimensional (hereinafter “2D”) training images of unique overhead objects within an environment; processes the 2D training images to determine training positions and projection points of each overhead object; trains one or more artificial intelligence (hereinafter “AI”) models to predict 2D positions of projection points of the overhead object; receives one or more 2D inference images of the overhead objects; processes the 2D inference images to determine one or more inferred positions of the overhead objects; predicts, via the trained AI models, 2D inferred positions of the projection points of the overhead objects; and provides them to one or more client devices. In some embodiments, the system alerts a person based on their proximity to the projection point of an overhead object.
1 . A method, comprising:
receiving one or more two-dimensional (2D) training images of one or more unique overhead objects within an environment;
processing the 2D training images to determine one or more training positions of each overhead object and one or more training positions of projection points of each overhead object;
based on at least the training positions of the overhead objects, training one or more artificial intelligence (AI) models to predict 2D positions of projection points of the overhead object;
receiving one or more 2D inference images of the overhead objects captured by one or more cameras within the environment;
processing the 2D inference images to determine one or more inferred positions of the overhead objects by:
identifying an overhead object in the one or more 2D inference images; and
placing a bounding box about the identified overhead object, the bounding box comprising the inferred positions;
predicting, via the one or more trained AI models and using the inferred positions of the overhead objects as input, 2D inferred positions of the projection points of the overhead objects; and
providing, to one or more client devices, the predicted 2D inferred positions of the projection points of the overhead objects.
2 . The method of claim 1 , wherein the 2D inference images further comprise one or more people in the environment, the method further comprising:
processing the 2D inference images to determine one or more inferred positions of the people in the environment.
3 . The method of claim 2 , wherein processing the 2D inference images to determine the inferred positions of the people in the environment is performed via one of:
amodal object detection, or instance segmentation.
4 . The method of claim 2 , further comprising:
determining that an inferred position of at least one of the people in the environment is within a specified proximity of at least one of the predicted 2D inferred positions of the projection points of the overhead objects; and
in response to the determination, providing an alert to at least one of the people in the environment warning of dangerous proximity to at least one of the overhead objects.
5 . The method of claim 4 , wherein the determination comprises determining that the distance between at least one of the predicted 2D inferred positions of the projection points of the overhead objects and at least one of the positions of the people in the environment is lower than a threshold distance.
6 . The method of claim 1 , further comprising:
normalizing the inferred positions of the people in the environment to a range between 0 and 1.
7 . The method of claim 1 , wherein the training positions of the overhead objects, the training positions of the projection points of the overhead objects, and the inferred positions of the overhead objects are configured to be pixel coordinates.
8 . The method of claim 1 , wherein training the one or more AI models comprises normalizing the training positions of the overhead objects and the training positions of the projection points of the overhead objects to a range between 0 and 1.
9 . The method of claim 8 , wherein training the one or more AI models further comprises applying coordinate transformation to the normalized training positions of the overhead objects and the normalized training positions of the projection points of the overhead objects, and wherein applying coordinate transformation comprises defining the normalized training positions of the overhead objects and the normalized training positions of the projection points of the overhead objects as relative to the bottom center of the bounding box for the overhead objects.
10 . The method of claim 1 , wherein the processing of the 2D training images and determining of the training positions of the overhead objects is performed via one or more of: amodal object detection, instance segmentation, and manual annotation.
11 . The method of claim 1 , wherein the processing of the 2D inference images and determining of the inferred positions of the overhead objects is performed via one of: amodal object detection, and instance segmentation.
12 . The method of claim 1 , wherein predicting the 2D inferred positions of the projection points of the overhead objects comprises normalizing the inferred positions of the overhead objects to a range between 0 and 1.
13 . The method of claim 12 , wherein predicting the 2D inferred positions of the projection points of the overhead objects comprises applying coordinate transformation to the normalized actual positions of the overhead objects, and wherein applying coordinate transformation comprises defining the normalized inferred positions of the overhead objects as relative to the bottom center of the bounding box for the overhead objects.
14 . The method of claim 1 , wherein the determined training positions of the overhead objects are configured to be 4 dimensional vectors, and wherein the determined training positions of the projection points of the overhead objects are configured to be 2 dimensional vectors.
15 . The method of claim 1 , wherein predicting the 2D inferred positions of the projection points of the overhead objects comprises using the inferred positions of the overhead objects as input to the one or more trained AI models, wherein the inferred positions of the overhead objects are configured to be 4 dimensional vectors, and wherein the predicted 2D inferred positions of the projection points of the overhead objects are configured to be 2 dimensional vectors.
16 . The method of claim 1 , wherein predicting the 2D inferred positions of the projection points of the overhead objects comprises converting a pixel representation of bounding boxes from a standard bounding box format to a format comprising a bottom-center x-axis, a bottom-center y-axis, a width, and a height.
17 . The method of claim 1 , wherein training the one or more AI models comprises comparing, via a loss function, the training positions of projection points of each overhead object to one or more specified actual positions of projection points of each overhead object.
18 . The method of claim 17 , wherein the loss function is a regression loss function comprising one of: a mean squared error (MSE) function, a mean absolute error (MAE) function, and a mean high-order error loss (MHOE) function.
19 . The method of claim 1 , wherein training the one or more AI models comprises:
feeding back one or more specified faulty predicted positions of projection points of the overhead objects to the one or more AI models to refine future output of the AI models.
20 . The method of claim 1 , wherein training the one or more AI models comprises:
obtaining an optimally performing AI model state by determining a state minimizing the specified faulty predicted positions.
21 . The method of claim 1 , wherein at least a subset of the received 2D training images is generated via a simulation, the simulation comprising at least a model of the overhead object.
22 . The method of claim 21 , wherein the simulation further comprises a model of each of the one or more cameras and one or more lens characteristics pertaining to the one or more cameras.
23 . The method of claim 1 , wherein the 2D inferred positions of the projection points of the overhead objects comprise a 2D inferred position of a center projection point of each of the overhead objects.
24 . The method of claim 23 , wherein the method is further configured to predict one or more additional 2D inferred positions of additional projection points of the overhead objects other than the center projection points of the overhead objects.
25 . The method of claim 1 , wherein each of the one or more cameras are configured to operate at any height and at any angle as long as the overhead object is within view of the camera, and wherein the ground level of the environment is visible within view of the camera.
26 . The method of claim 1 , further comprising:
wherein the inferred positions of the people are projection points of people in the environment;
inputting the determined position into a proximity based alert analyzer; and
determining, by the proximity based alert analyzer, based on a distance threshold limit, whether there is a close proximity between the people within the environment and the overhead objects within the environment.
27 . The method of claim 26 , wherein the proximity based alert analyzer measures the distance between each projection point of each overhead objects and each projection point of each person in the area.
28 . The method of claim 27 , wherein each projection point of each person are feet projection points of each person.
29 . A method, comprising:
receiving one or more two-dimensional (2D) training images of one or more unique overhead objects within an environment;
processing the 2D training images to determine one or more training positions of each overhead object and one or more training positions of projection points of each overhead object;
based on at least the training positions of the overhead objects, training one or more artificial intelligence (AI) models to predict 2D positions of projection points of the overhead object;
receiving one or more 2D inference images of the overhead objects captured by one or more cameras within the environment;
processing the 2D inference images to determine one or more inferred positions of the overhead objects;
predicting, via the one or more trained AI models and using the inferred positions of the overhead objects as input, 2D inferred positions of the projection points of the overhead objects; and
providing, to one or more client devices, the predicted 2D inferred positions of the projection points of the overhead objects;
wherein training the one or more AI models comprises normalizing the training positions of the overhead objects and the training positions of the projection points of the overhead objects to a range between 0 and 1, and wherein training the one or more AI models further comprises applying coordinate transformation to the normalized training positions of the overhead objects and the normalized training positions of the projection points of the overhead objects, and wherein applying coordinate transformation comprises defining the normalized training positions of the overhead objects and the normalized training positions of the projection points of the overhead objects as relative to the bottom center of a bounding box for the overhead objects.
30 . A method, comprising:
receiving one or more two-dimensional (2D) training images of one or more unique overhead objects within an environment;
processing the 2D training images to determine one or more training positions of each overhead object and one or more training positions of projection points of each overhead object;
based on at least the training positions of the overhead objects, training one or more artificial intelligence (AI) models to predict 2D positions of projection points of the overhead object;
receiving one or more 2D inference images of the overhead objects captured by one or more cameras within the environment;
processing the 2D inference images to determine one or more inferred positions of the overhead objects;
predicting, via the one or more trained AI models and using the inferred positions of the overhead objects as input, 2D inferred positions of the projection points of the overhead objects; and
providing, to one or more client devices, the predicted 2D inferred positions of the projection points of the overhead objects;
wherein predicting the 2D inferred positions of the projection points of the overhead objects comprises normalizing the inferred positions of the overhead objects to a range between 0 and 1, and wherein predicting the 2D inferred positions of the projection points of the overhead objects comprises applying coordinate transformation to the normalized actual positions of the overhead objects, and wherein applying coordinate transformation comprises defining the normalized inferred positions of the overhead objects as relative to the bottom center of a bounding box for the overhead objects.