Intent-based dynamic change of compute resources of vehicle perception system
The present disclosure provides perception system for a vehicle that includes a plurality of imaging devices for producing images of an environment of the vehicle; a perception filter for receiving the images produced by the imaging devices, wherein the perception filter determines compute resource priority instructions based on an intent of the vehicle and a current state of the vehicle; and a compute module for receiving the compute resource priority instructions from the perception filter and allocating compute resources among the imaging devices in accordance with the compute resource priority instructions.
1. A perception system for a vehicle, the perception system comprising:
a plurality of imaging devices for producing image data comprising images of an environment of the vehicle;
a perception filter for receiving the image data produced by the imaging devices, wherein the perception filter determines compute resource priority instructions based on an intent of the vehicle and a current state of the vehicle received by the perception filter, wherein the intent of the vehicle is based on a planned route of the vehicle and wherein the current state of the vehicle includes at least a current position and a velocity of the vehicle, the perception filter determining a region of interest (“ROI”) for the vehicle based on the intent of the vehicle and the current state of the vehicle and wherein the ROI is used by the perception filter to determine the compute resource priority instructions and crop and filter the received image data based on the intent of the vehicle and the current state of the vehicle;
a compute module for receiving the compute resource priority instructions from the perception filter and allocating a greater amount of compute resources to a first one of the imaging devices than to a second one of the imaging devices to enable the first one of the imaging devices to produce higher quality image data than the second one of the imaging devices, wherein the first one of the imaging devices is directed toward and produces images of the ROI and the second one of the imaging devices is not directed toward and does not produce images of the ROI thereby increasing operational quality and decreasing, latency of the perception system; and
a perception module for receiving at least one of the cropped and filtered image data from current the perception filter and perceiving the environment of the vehicle based on the received at least one of the cropped and filtered image data;
wherein the vehicle is caused to take appropriate action based on the perceived environment, the current state of the vehicle, and the intent of the vehicle; and
wherein the perception filter further provides scanning instructions to at least one of the imaging devices.
2. The perception system of claim 1 , wherein the ROI is used by the perception filter to determine a resolution of the selected at least one of the received images of the environment of the vehicle.
3. The perception system of claim 1 , wherein the vehicle comprises an autonomous vehicle.
4. The perception system of claim 1 , wherein the imaging devices comprise at least one of a camera, a LIDAR device, and a RADAR device.
5. A vehicle comprising:
an onboard computer;
a sensor suite comprising a plurality of sensors and
a perception system including:
a plurality of imaging devices for producing image data comprising images of an environment of the vehicle;
a perception filter for receiving the image data produced by the imaging devices, wherein the perception filter determines compute resource priority instructions based on an intent of the vehicle and a current state of the vehicle received by the perception filter, wherein the intent of the vehicle is based on a planned route of the vehicle and wherein the current state of the vehicle includes at least a current position and a velocity of the vehicle, the perception filter determining a region of interest (“ROI”) for the vehicle based on the intent of the vehicle and the current state of the vehicle and wherein the ROI is used by the perception filler to determine the compute resource priority instructions and crop and filter the received image data based on the intent of the vehicle and the current state of the vehicle;
a compute module for receiving the compute resource priority instructions from the perception filter and allocating a greater amount of compute resources to a first one of the imaging devices than to a second one of the imaging devices to enable the first one of the imaging devices to produce higher quality image data than the second one of the imaging devices, wherein the first one of the imaging devices is directed toward and produces images of the ROI and the second one of the imaging devices is not directed toward and does not produce images of the ROI, thereby increasing operational quality and decreasing latency of the perception system; and
a perception module for receiving at least one of the cropped and filtered image data from current the perception filter and perceiving the environment of the AV based on the received at least one of the cropped and filtered image data;
wherein the vehicle is caused to take an appropriate action based on the perceived environment, the current state of the vehicle, and the intent of the vehicle; and
wherein the perception filter further provides scanning, instructions to at least one of the imaging devices.
6. The vehicle of claim 5 , wherein the ROI is used by the perception filter to determine a resolution of the selected at least one of the received images.
7. A method comprising:
deploying a plurality of imaging devices for producing image data comprising images of an environment of a vehicle;
determining a current state of the vehicle and an intent of the vehicle, wherein the intent of the vehicle is based on a planned route of the vehicle and wherein the current state of the vehicle includes at least a current position and a velocity of the vehicle;
receiving by a perception filter the image data produced by the imaging devices, wherein the perception filter determines compute resource priority instructions based on the intent of the vehicle and the current state of the vehicle, the perception filter determining a region of interest (“ROI”) for the vehicle based on the intent of the vehicle and the current state of the vehicle and wherein the ROI is used by the perception filter to determine the compute resource priority instructions and crop and filter the received image data based on the intent of the vehicle and the current state of the vehicle;
receiving by a compute module the compute resource priority instructions; and
allocating by the compute module compute a greater amount of compute resources to a first one of the imaging devices than to a second one of the imaging devices to enable the first one of the imaging devices to produce higher quality image data than the second one of the imaging devices, wherein the first one of the imaging devices is directed toward and produces images of the ROI and the second one of the imaging devices is not directed toward and does not produce images of the ROI, thereby increasing operational quality and decreasing latency of the perception system;
receiving by a perception module at least one of the cropped and filtered image data from current the perception filter and perceiving the environment of the vehicle based on the received at least one of the cropped and filtered image data;
causing the vehicle to take appropriate action based on the perceived environment, the current state of the vehicle, and the intent of the vehicle; and
providing scanning instructions to at least one of the imaging devices.
8. The perception system of claim 1 , wherein the current state of the vehicle comprises at least one of a current speed of the vehicle, a current location of the vehicle, and a current direction of travel of the vehicle.
9. The vehicle of claim 5 , wherein the current state of the vehicle comprises at least one of a current speed of the vehicle, a current location of the vehicle, and a current direction of travel of the vehicle.
10. The vehicle of claim 5 , wherein the vehicle comprises an autonomous vehicle.
11. The vehicle of claim 5 , wherein the imaging devices comprise at least one of a camera, a LIDAR device, and a RADAR device.
12. The method of claim 7 , wherein the current state of the vehicle comprises at least one of a current speed of the vehicle, a current location of the vehicle, and a current direction of travel of the vehicle.
13. The method of claim 7 further comprising the perception filter using the ROI to determine a resolution of the selected at least one of the received images.
14. The method of claim 7 , wherein the vehicle comprises an autonomous vehicle.
15. The method of claim 7 , wherein the imaging devices comprise at least one of a camera, a LIDAR device, and a RADAR device.