Inferring future visibility to improve autonomous vehicle driving behavior and safety
A system includes a memory storing instructions and a processing device operatively coupled to the memory, wherein the instructions, when executed by the processing device, cause the processing device to perform operations including obtaining a set of input data representing a set of objects within a driving environment being navigated by an autonomous vehicle (AV), generating, based on the set of input data, future visibility information for a set of future locations along a trajectory of the AV within the driving environment, determining, based on the future visibility information, whether a point of sufficient visibility (PSV) exists among the set of future locations, wherein the PSV is a point determined to be clear of occlusions with respect to a speculative object, and controlling operation of the AV based on the PSV determined to exist among the set of future locations.
1 . A system comprising:
a memory storing instructions; and
a hardware processor operatively coupled to the memory, wherein the instructions, when executed by the hardware processor, cause the hardware processor to perform operations comprising:
obtaining a set of input data representing a set of objects within a driving environment being navigated by an autonomous vehicle (AV), the set of input data being associated with a first time;
generating, based on the set of input data associated with the first time, future visibility information for a plurality of future locations along a trajectory of the AV within the driving environment, wherein the future visibility information for each future location of the plurality of future locations corresponds to a second time during which visibility of the driving environment is likely to change, and wherein the second time for each future location is a respective amount of time later than the first time;
determining, based on the future visibility information, whether a point of sufficient visibility (PSV) exists among the plurality of future locations, wherein the PSV is identified, from among the plurality of future locations, as a first future location along the trajectory of the AV that is determined to be clear of occlusions with respect to a speculative object; and
controlling operation of the AV based on the PSV determined to exist among the plurality of future locations.
2 . The system of claim 1 , wherein the set of objects comprises a set of permanent objects within the driving environment and a set of perception objects observed by the AV within the driving environment.
3 . The system of claim 2 , wherein obtaining the set of input data comprises obtaining a surfel map corresponding to the set of permanent objects.
4 . The system of claim 1 , wherein determining, based on the future visibility information, whether the PSV exists among the plurality of future locations further comprises:
identifying a region of interest (ROI) with respect to a viewpoint of the AV;
determining, based on the ROI, a set of future visibilities associated with the plurality of future locations, wherein each future location of the plurality of future locations corresponds to a respective future visibility of the set of future visibilities; and
determining whether the PSV exists within the plurality of future locations based on the set of future visibilities.
5 . The system of claim 4 , wherein determining the set of future visibilities further comprises determining, for each future location of the plurality of future locations, whether the ROI is occluded using raycasting by:
identifying a number of occluded rays that exist with respect to the ROI;
determining whether the number of occluded rays satisfies a threshold condition; and
in response to determining that the number of occluded rays satisfies the threshold condition, identifying the ROI as being occluded.
6 . The system of claim 1 , wherein controlling operation of the AV based on the PSV further comprises identifying at least one of: a yield location, a distance to the PSV, or a time to the PSV.
7 . The system of claim 1 , wherein the operations further comprise, in response to determining that the PSV does not exist among the plurality of future locations, using a set of default settings to control operation of the AV.
8 . A method comprising:
obtaining, by a hardware processor, a set of input data representing a set of objects within a driving environment being navigated by an autonomous vehicle (AV), the set of input data being associated with a first time;
generating, by the hardware processor based on the set of input data associated with the first time, future visibility information for a plurality of future locations along a trajectory of the AV within the driving environment, wherein the future visibility information for each future location of the plurality of future locations corresponds to a second time during which visibility of the driving environment is likely to change, and wherein the second time for each future location is a respective amount of time later than the first time;
determining, by the hardware processor based on the future visibility information, whether a point of sufficient visibility (PSV) exists among the plurality of future locations, wherein the PSV is identified, from among the plurality of future locations, as a first future location along the trajectory of the AV that is a point determined to be clear of occlusions with respect to a speculative object; and
controlling, by the hardware processor, operation of the AV based on the PSV determined to exist among the plurality of future locations.
9 . The method of claim 8 , wherein the set of objects comprises a set of permanent objects within the driving environment and a set of perception objects observed by the AV within the driving environment.
10 . The method of claim 9 , wherein obtaining the set of input data comprises obtaining a surfel map corresponding to the set of permanent objects.
11 . The method of claim 8 , wherein determining, based on the future visibility information, whether the PSV exists among the plurality of future locations further comprises:
identifying a region of interest (ROI) with respect to a viewpoint of the AV;
determining, based on the ROI, a set of future visibilities associated with the plurality of future locations, wherein each future location of the plurality of future locations corresponds to a respective future visibility of the set of future visibilities; and
determining whether the PSV exists within the plurality of future locations based on the set of future visibilities.
12 . The method of claim 11 , wherein determining the set of future visibilities further comprises determining, for each future location of the plurality of future locations, whether the ROI is occluded using raycasting by:
identifying a number of occluded rays that exist with respect to the ROI;
determining whether the number of occluded rays satisfies a threshold condition; and
in response to determining that the number of occluded rays satisfies the threshold condition, identifying the ROI as being occluded.
13 . The method of claim 8 , wherein controlling operation of the AV based on the PSV further comprises identifying at least one of: a yield location, a distance to the PSV, or a time to the PSV.
14 . The method of claim 8 , further comprising, in response to determining that the PSV does not exist among the plurality of future locations, using, by the hardware processor, a set of default settings to control operation of the AV.
15 . A non-transitory computer-readable storage medium having instructions stored thereon that, when executed by a hardware processor, cause the hardware processor to perform operations comprising:
obtaining a set of input data representing a set of objects within a driving environment being navigated by an autonomous vehicle (AV), the set of input data being associated with a first time;
generating, based on the set of input data associated with the first time, future visibility information for a plurality of future locations along a trajectory of the AV within the driving environment, wherein the future visibility information for each future location of the plurality of future locations corresponds to a second time during which visibility of the driving environment is likely to change, and wherein the second time for each future location is a respective amount of time later than the first time;
determining, based on the future visibility information, whether a point of sufficient visibility (PSV) exists among the plurality of future locations, wherein the PSV is identified, from among the plurality of future locations, as a first future location along the trajectory of the AV that is determined to be clear of occlusions with respect to a speculative object; and
controlling operation of the AV based on the PSV determined to exist among the plurality of future locations.
16 . The non-transitory computer-readable storage medium of claim 15 , wherein the set of objects comprises a set of permanent objects within the driving environment and a set of perception objects observed by the AV within the driving environment, and wherein obtaining the set of input data comprises obtaining a surfel map corresponding to the set of permanent objects.
17 . The non-transitory computer-readable storage medium of claim 15 , wherein determining, based on the future visibility information, whether the PSV exists among the plurality of future locations further comprises:
identifying a region of interest (ROI) with respect to a viewpoint of the AV;
determining, based on the ROI, a set of future visibilities associated with the plurality of future locations, wherein each future location of the plurality of future locations corresponds to a respective future visibility of the set of future visibilities; and
determining whether the PSV exists within the plurality of future locations based on the set of future visibilities.
18 . The non-transitory computer-readable storage medium of claim 17 , wherein determining the set of future visibilities further comprises determining, for each future location of the plurality of future locations, whether the ROI is occluded using raycasting by:
identifying a number of occluded rays that exist with respect to the ROI;
determining whether the number of occluded rays satisfies a threshold condition; and
in response to determining that the number of occluded rays satisfies the threshold condition, identifying the ROI as being occluded.
19 . The non-transitory computer-readable storage medium of claim 15 , wherein controlling operation of the AV based on the PSV further comprises identifying at least one of: a yield location, a distance to the PSV, or a time to the PSV.
20 . The non-transitory computer-readable storage medium of claim 15 , wherein the operations further comprise, in response to determining that the PSV does not exist among the plurality of future locations, using a set of default settings to control operation of the AV.