Filtering of dynamic objects from vehicle generated map
A method and system for a vehicle control system generates maps utilized for charting a path of a vehicle through an environment. The method performed by the system obtains information indicative of vehicle movement from at least one vehicle system and images including objects within an environment from a camera mounted on the vehicle. The system uses the gathered information and images to create a depth map of the environment. The system also generates an image point cloud map from images taken with a vehicle camera and a radar point cloud map with velocity information from a radar sensor mounted on the vehicle. The depth map and the point cloud maps are fused together and any dynamic objects filtered out from the final map used for operation of the vehicle.
1 . A method of creating a map of an environment surrounding a vehicle comprising:
obtaining, by a controller processor, information indicative of vehicle movement from at least one vehicle system, wherein the vehicle system comprises at least one of a wheel speed sensor, an acceleration sensor, an inertial measurement unit or a global positioning system;
obtaining, by the controller processor, images including objects within the environment surrounding the vehicle from a mono-camera mounted on the vehicle;
determining a pose of the mono-camera in a local reference frame;
creating, by the controller processor, a depth map of the environment surrounding the vehicle based on images obtained from the mono-camera, the pose of the mono-camera, and the information indicative of vehicle movement, the depth map associating a distance to the vehicle for each pixel in the images;
creating, by the controller processor, an image point cloud map of the environment surrounding the moving vehicle based on the depth map and based on the images obtained from the mono-camera mounted to the moving vehicle;
creating, by the controller processor, a radar point cloud map of object velocities with information from a radar sensor mounted on the vehicle and the information indicative of vehicle movement, wherein the radar point cloud map includes a plurality of points that are indicative of a relative velocity between the moving vehicle and the objects around the moving vehicle;
creating, by the controller processor, a first occupancy grid of the image point cloud map, and creating a second occupancy grid of the radar point cloud map;
creating, by the controller processor, a fused map by combining the first occupancy grid with the second occupancy grid and removing any dynamic objects from the fused map;
at least one of using, by the controller processor or another processor in the vehicle, the fused map for navigating the vehicle autonomously or semi-autonomously, or communicating, by the controller processor, the fused map to a vehicle driving control system of the vehicle for autonomously or semi-autonomously controlling the vehicle.
2 . The method as recited in claim 1 , further comprising identifying, by the controller processor, an object as a dynamic object in response to a cluster of points within the radar point cloud map having a velocity that indicates movement relative to static features within the environment.
3 . The method as recited in claim 1 , further comprising using, by the controller processor, the pose of the mono-camera in the creation of at least one of the image point cloud map or the radar point cloud map.
4 . The method as recited in claim 1 , wherein the depth map includes points that are indicative of a distance between the vehicle and objects surrounding the vehicle.
5 . The method as recited in claim 1 , wherein the object velocities comprise a Doppler velocity.
6 . The method as recited in claim 1 , wherein the information indicative of vehicle movement is generated with a vehicle dynamic model.
7 . The method as recited in claim 1 , wherein the image point cloud map and the radar point cloud map are each in three dimensions, and wherein the occupancy grid of the image point cloud map and the occupancy grid of the radar point cloud map are each in two dimensions.
8 . An autonomous vehicle system for creating a map of static objects within an environment surrounding the autonomous vehicle, the system comprising:
a controller processor configured to:
obtain information indicative of vehicle movement from vehicle navigation system comprising at least one of a wheel speed sensor, an acceleration sensor, an inertial measurement unit or a global positioning system;
obtain images of objects within the environment surrounding vehicle from a mono-camera mounted on the vehicle;
determine a pose of the mono-camera in a local reference frame;
create a depth map of the environment surrounding the vehicle based on images obtained from the mono-camera, the pose of the mono-camera and the information indicative of vehicle movement, the depth map associating a distance to the vehicle for each pixel in the images;
create an image point cloud map of the environment surrounding the moving vehicle based on the depth map and based on the images obtained from the mono-camera mounted to the moving vehicle;
create a radar point cloud map of object velocities with information from a radar sensor mounted on the vehicle and the information indicative of vehicle movement, wherein the radar point cloud map includes a plurality of points that are indicative of a relative velocity between the moving vehicle and the objects around the moving vehicle;
create a fused map by combining the occupancy grid of the image point cloud map with the occupancy grid of the radar point cloud map, and remove any dynamic objects; and
at least one of use the fused map to provide navigation for the vehicle or communicate the fused map to a vehicle driving control system of the vehicle for autonomously or semi-autonomously controlling the vehicle.
9 . The autonomous vehicle system as recited in claim 8 , wherein the controller processor is further configured to identify an object as a dynamic object in response to a cluster of points within the radar point cloud map having a velocity that indicates movement relative to static features within the environment.
10 . The autonomous vehicle system as recited in claim 8 , including a data storage medium that includes instructions executable by the controller processor.
11 . A non-transitory computer readable medium comprising stored instructions executable by a controller processor for creating a map of an environment surrounding a vehicle, the instructions comprising:
instructions prompting a controller processor to obtain images including objects within the environment surrounding the vehicle from a mono-camera mounted on the vehicle;
instructions prompting a controller processor to determine a pose of the mono-camera in a local reference frame;
instructions prompting a controller processor to create a depth map of the environment surrounding the vehicle based on the images obtained from the mono-camera, on information indicative of the vehicle movement from a navigation system of the vehicle comprising at least one of a wheel speed sensor, an acceleration sensor, a inertial measurement unit or a global positioning system, and on the pose of the mono-camera, the depth map associating a distance to the vehicle for each pixel in the images;
instructions prompting a controller processor to create an image point cloud map of the environment surrounding the moving vehicle based on the depth map and based on the images obtained from the mono-camera mounted to the moving vehicle;
instructions prompting the controller processor to create a radar point cloud map of object velocities with information from a radar sensor and the information indicative of vehicle movement, wherein the radar point cloud map includes a plurality of points that are indicative of a relative velocity between the moving vehicle and the objects around the moving vehicle;
instructions prompting the controller processor to create an occupancy grid of the image point cloud map, and create an occupancy grid of the radar point cloud map;
instructions prompting the controller processor to create a fused map by combining the occupancy grid of the image point cloud map with the occupancy grid of the radar point cloud map, and to remove any dynamic objects from the fused map; and
instructions prompting the controller processor to at least one of use the fused map to provide navigation for the vehicle or communicate the fused map to a vehicle driving control system of the vehicle for autonomously or semi-autonomously controlling the vehicle.
12 . The non-transitory computer readable medium as recited in claim 11 , further including instructions for prompting the controller processor to identify an object as a dynamic object in response to a cluster of points within the radar point cloud map having a velocity that indicates movement relative to static features within the environment.
13 . The non-transitory computer readable medium as recited in claim 11 , wherein the non-transitory computer readable medium comprises at least one of a volatile memory or nonvolatile memory.
14 . The non-transitory computer readable medium as recited in claim 11 , wherein the image point cloud map and the radar point cloud map are each in three dimensions, and wherein the occupancy grid of the image point cloud map and the occupancy grid of the radar point cloud map are each in two dimensions.