Enhanced user interface generation for parking based on occupancy machine learning models
Systems and methods for enhanced user interface generation for parking based on occupancy machine learning model. An example method includes obtaining images from a multitude of image sensors positioned about a vehicle; computing a forward pass through an occupancy network to output, at least, information reflecting, for individual angular ranges about the vehicle, whether an object is within a threshold distance of the vehicle for an individual range along with an estimated distance to the object; and causing presentation, via a display of the vehicle, of a user interface depicting a graphical representation of the vehicle and the output information.
1 . A system comprising:
one or more processors configured to:
receive data associated with a first image and a second image, the data associated with the first image and the second image generated by one or more sensors of an ego;
determine a plurality of distances to one or more objects relative to the ego based on the first image, the second image, and an occupancy network, the occupancy network configured to receive the data associated with the first image and the second image as input and, in response to execution of the occupancy network on the first image and the second image, output the plurality of distances to the one or more objects relative to the ego;
generate a graphical user interface based on the plurality of distances to the one or more objects relative to the ego, the graphical user interface comprising a representation of the ego, a contour interposed between the representation of the ego and the one or more objects, and the one or more objects positioned relative to the ego;
provide data associated with the graphical user interface to at least one processor associated with a display screen, the data associated with the graphical user interface configured to cause the at least one processor to display the graphical user interface on the display screen;
and control operation of the ego based on the plurality of distances to the one or more objects.
2 . The system of claim 1 , wherein the one or more processors configured to generate the graphical user interface are configured to:
determine the contour, where the contour indicates a relationship of a portion of each of the one or more objects positioned relative to the ego;
determine a placement of the contour relative to respective objects of the one or more objects positioned relative to the ego; and
generate the graphical user interface based on the contour and the placement of the contour relative to the respective objects.
3 . The system of claim 2 , wherein the one or more processors configured to determine the contour are configured to:
determine the contour of the portion of each of the one or more objects based on the distances of the plurality of distances corresponding to each object of the one or more objects.
4 . The system of claim 3 , wherein the one or more processors configured to determine the contour are configured to:
determine a contour profile for the contour of the portion of each of the one or more objects positioned relative to the ego based on the plurality of distances corresponding to each object of the one or more objects.
5 . The system of claim 3 , wherein the one or more processors configured to determine the contour are configured to:
determine a contour profile for the contour of the portion of each of the one or more objects positioned relative to the ego based on the plurality of distances corresponding to each object of the one or more objects, each contour profile represented as a gradient.
6 . The system of claim 3 , wherein the one or more processors configured to determine the plurality of distances to one or more objects relative to the ego are configured to:
determine a subset of distances of the plurality of distances satisfies a threshold range of distances, and
wherein the one or more processors configured to generate the graphical user interface based on the plurality of distances to the one or more objects relative to the ego are configured to:
generate the graphical user interface based on the plurality of distances to the one or more objects that satisfy the threshold range of distances.
7 . The system of claim 6 , wherein the one or more processors configured to generate the graphical user interface based on the plurality of distances to the one or more objects relative to the ego are configured to:
forgo generating the graphical user interface based on the plurality of distances to the one or more objects that do not satisfy the threshold range of distances.
8 . The system of claim 3 , wherein the one or more processors configured to determine the plurality of distances to one or more objects relative to the ego are configured to:
determine a height for each distance of the plurality of distances based on the first image and the second image; and
determine that the height for each distance of the plurality of distances satisfies a threshold range of heights, and
wherein the one or more processors configured to generate the graphical user interface based on the plurality of distances to the one or more objects relative to the ego are configured to:
generate the graphical user interface based on the plurality of distances to the one or more objects that are associated with heights that satisfy the threshold range of heights.
9 . The system of claim 8 , wherein the one or more processors configured to generate the graphical user interface based on the plurality of distances to the one or more objects that are associated with heights that satisfy the threshold range of heights are configured to:
forgo generating the graphical user interface based on the plurality of distances to the one or more objects that are associated with heights that do not satisfy the threshold range of heights.
10 . A method, comprising:
receiving, by at least one processor, data associated with a first image and a second image, the data associated with the first image and the second image generated by one or more sensors of an ego;
determining, by the at least one processor, a plurality of distances to one or more objects relative to the ego based on the first image and the second image based on an output generated by an occupancy network in response to execution of the occupancy network on the first image and the second image;
generating, by the at least one processor, a graphical user interface based on the plurality of distances to the one or more objects relative to the ego, the graphical user interface comprising a representation of the ego, a contour interposed between the representation of the ego and the one or more objects, and the one or more objects positioned relative to the ego;
providing, by the at least one processor, data associated with the graphical user interface to at least one processor associated with a display screen, the data associated with the graphical user interface configured to cause the at least one processor to display the graphical user interface on the display screen; and
controlling, by the at least one processor, operation of the ego based on the plurality of distances to the one or more objects.
11 . The method of claim 10 , wherein generating the graphical user interface comprises:
determining, by the at least one processor, a contour of a portion of each of the one or more objects positioned relative to the ego;
determining, by the at least one processor, a placement of the contour relative to respective objects of the one or more objects positioned relative to the ego; and
generating, by the at least one processor, the graphical user interface based on the contour and the placement of the contour relative to the respective objects.
12 . The method of claim 11 , wherein determining the contour comprises:
determining, by the at least one processor, the contour of the portion of each of the one or more objects based on the distances of the plurality of distances corresponding to each object of the one or more objects.
13 . The method of claim 12 , wherein determining the contour comprises:
determining, by the at least one processor, a contour profile for the contour of the portion of each of the one or more objects positioned relative to the ego based on the plurality of distances corresponding to each object of the one or more objects.
14 . The method of claim 12 , wherein determining the contour comprises:
determining, by the at least one processor, a contour profile for the contour of the portion of each of the one or more objects positioned relative to the ego based on the plurality of distances corresponding to each object of the one or more objects, each contour profile represented as a gradient.
15 . The method of claim 12 , wherein determining the plurality of distances to one or more objects relative to the ego comprises:
determining, by the at least one processor, a subset of distances of the plurality of distances satisfies a threshold range of distances, and
wherein generating the graphical user interface based on the plurality of distances to the one or more objects relative to the ego comprises:
generating, by the at least one processor, the graphical user interface based on the plurality of distances to the one or more objects that satisfy the threshold range of distances.
16 . The method of claim 15 , wherein generating the graphical user interface based on the plurality of distances to the one or more objects relative to the ego comprises:
forgoing generating the graphical user interface based on the plurality of distances to the one or more objects that do not satisfy the threshold range of distances.
17 . The method of claim 12 , wherein determining the plurality of distances to one or more objects relative to the ego comprises:
determining, by the at least one processor, a height for each distance of the plurality of distances based on the first image and the second image; and
determining, by the at least one processor, that the height for each distance of the plurality of distances satisfies a threshold range of heights, and
wherein generating the graphical user interface based on the plurality of distances to the one or more objects relative to the ego comprises:
generating, by the at least one processor, the graphical user interface based on the plurality of distances to the one or more objects that are associated with heights that satisfy the threshold range of heights.
18 . The method of claim 17 , wherein generating the graphical user interface based on the plurality of distances to the one or more objects that are associated with heights that satisfy the threshold range of heights comprises:
forgoing, by the at least one processor, generating the graphical user interface based on the plurality of distances to the one or more objects that are associated with heights that do not satisfy the threshold range of heights.
19 . A method, comprising:
obtaining, by at least one processor, images from a multitude of image sensors positioned about a vehicle;
computing, by the at least one processor, a forward pass through an occupancy network to output, at least, information reflecting, for individual angular ranges about the vehicle, whether an object is within a threshold distance of the vehicle for an individual range along with an estimated distance to the object;
in response to execution of the occupancy network, causing, by the at least one processor, presentation, via a display of the vehicle, of a user interface depicting a graphical representation of the vehicle and the information reflecting whether the object is within a threshold distance of the vehicle as a contour interposed between the graphical representation of the vehicle and a graphical representation of the object; and
controlling, by the at least one processor, operation of the vehicle based on whether the object is within the threshold distance.
20 . The method of claim 19 , wherein the graphical representation of the output information includes a contour positioned about the vehicle.