Detection and filtering of abnormal sensor data for object detection in automotive applications
An object detection system of a vehicle includes a camera and a LIDAR sensor. The camera and the LIDAR sensor sense an environment to generate an image and a point cloud that depict the environment. The image and point cloud are preprocessed to facilitate comparison between the image and the point cloud. Similarity between the image and the point cloud in depicting the environment is determined to detect abnormal sensor data. Abnormal sensor data is further detected based on directional pattern strengths of edges of the image and expanded points of the point cloud. Detected abnormal sensor data in the image and point cloud are filtered to generate a secure image and a secure point cloud, which are provided to a perception engine to detect objects or other features in the environment.
1. A method of filtering abnormal sensor data for an object detection system of a vehicle, the method comprising:
generating a preprocessed image by preprocessing a raw image received from a camera of the object detection system;
generating a preprocessed point cloud by preprocessing a raw point cloud received from a LIDAR sensor of the object detection system;
determining a similarity between a region of the preprocessed image and a region of the preprocessed point cloud;
in response to detecting that the region of the preprocessed image is not similar to the region of the preprocessed point cloud, generating a secure raw image by filtering image data of the raw image that corresponds to the region of the preprocessed image;
in response to detecting that the region of the preprocessed image is not similar to the region of the preprocessed point cloud, generating a secure raw point cloud by filtering point cloud data of the raw point cloud that corresponds to the region of the preprocessed point cloud;
detecting an object in a sensed environment using the secure raw image and the secure raw point cloud.
2. The method of claim 1 , wherein generating the preprocessed image and generating the preprocessed point cloud comprises:
converting the raw image into a gradient image;
expanding points of the raw point cloud to generate an expanded point cloud; and
dividing the gradient image and the expanded point cloud into a plurality of corresponding grids.
3. The method of claim 2 , wherein the region of the of the preprocessed image is a grid of the preprocessed image, and the region of the preprocessed point cloud is a grid of the preprocessed point cloud.
4. The method of claim 3 , wherein determining the similarity between the region of the preprocessed image and the region of the preprocessed point cloud comprises:
calculating a similarity score for the grid of the preprocessed image and the grid of the preprocessed point cloud;
weighing the similarity score to generate a weighted similarity score; and
comparing the weighted similarity score to a threshold.
5. The method of claim 4 , wherein weighing the similarity score includes adjusting the similarity score by a predetermined value.
6. An object detection system of a vehicle, the object detection system comprising:
a camera that generates an image of an environment that is forward of the vehicle;
a LIDAR sensor that generates a point cloud of the environment; and
a data inspector comprising a memory and a processor, the memory storing instructions that when executed by the processor cause the data inspector to:
determine a similarity between the image and the point cloud in depicting a first region of the environment;
in response to detecting that the image and the point cloud are not similar in depicting the first region of the environment, filter a first portion of the image and a first portion of the point cloud that depict the first region of the environment to generate a secure image and a secure point cloud; and
provide the secure image and the secure point cloud to a perception engine to detect one or more objects in the environment.
7. The object detection system of claim 6 , wherein the instructions stored in the memory of the data inspector, when executed by the processor of the data inspector, cause the data inspector to determine the similarity between the image and the point cloud in depicting the first region of the environment by:
converting the image to a gradient image;
horizontally expanding the point cloud to generate an expanded point cloud;
dividing the gradient image and the expanded point cloud into a plurality of corresponding grids; and
determining a similarity between a first grid of the gradient image and a first grid of the expanded point cloud that correspond to the first region of the environment.
8. The object detection system of claim 7 , wherein the instructions stored in the memory of the data inspector, when executed by the processor of the data inspector, further cause the data inspector to:
detect a directional pattern strength of a second grid of the gradient image;
detect a directional pattern strength of a second grid of the expanded point cloud, the second grids of the gradient image and the expanded point cloud depicting a second region of the environment;
compare the directional pattern strengths of the second grids of the gradient image and the expanded point cloud to a regression curve; and
detect that the second grids of the gradient image and the expanded point cloud are abnormal in response to the directional pattern strengths of the second grids of the gradient image and the expanded point cloud not being within a predetermined distance of the regression curve.
9. The object detection system of claim 8 , wherein the instructions stored in the memory of the data inspector, when executed by the processor of the data inspector, further cause the data inspector to:
filter a second portion of the image and a second portion of the point cloud that depict the second region of the environment to generate the secure image and the secure point cloud.