Lidar-based classification of object movement
View Patent ↗Within machine vision, object movement is often estimated by applying image evaluation techniques to visible light images, utilizing techniques such as perspective and parallax. However, the precision of such techniques may be limited due to visual distortions in the images, such as glare and shadows. Instead, lidar data may be available (e.g., for object avoidance in automated navigation), and may serve as a high-precision data source for such determinations. Respective lidar points of a lidar point cloud may be mapped to voxels of a three-dimensional voxel space, and voxel clusters may be identified as objects. The movement of the lidar points may be classified over time, and the respective objects may be classified as moving or stationary based on the classification of the lidar points associated with the object. This classification may yield precise results, because voxels in three-dimensional voxel space present clearly differentiable statuses when evaluated over time.
1. A method comprising:
executing on one or more processors, instructions configured to cause the processor to:
receive, from a lidar detector, at least a first plurality and a second plurality of reflected lidar points, each plurality of lidar points forming a lidar point cloud;
map the lidar points in each lidar point cloud to a voxel in a three-dimensional voxel space;
identify at least one object represented by a voxel cluster of voxels sharing an adjacency within the three-dimensional voxel space, wherein adjacency of two voxels is determined by a specified number of voxels between said two voxels;
for each of the plurality of lidar points in each lidar point cloud:
associate the lidar point with the identified object, and
classify the movement of the lidar point according to the identified object; and
for each identified object, classify the movement of the object according to the movement of the respective lidar points associated with the object.
2. The method of claim 1 wherein:
a first axis within the three-dimensional voxel space has a first adjacency threshold;
a second axis within the three-dimensional voxel space has a second adjacency threshold that is different from the first adjacency threshold; and
identifying the object further comprises identifying the object represented by a cluster of voxels respectively sharing, for respective axes within the three-dimensional space, an adjacency within the adjacency threshold of the axis.
3. The method of claim 1 , wherein identifying the at least one object further comprises:
identifying at least two voxel clusters of voxels within the three-dimensional voxel space; and
determining that the at least two voxel clusters are connected.
4. The method of claim 3 , wherein determining that the at least two voxel clusters are connected further comprises:
selecting a first voxel cluster; and
performing a nearest neighbor search of at least one other voxel cluster to identify a connection with the first voxel cluster.
5. The method of claim 1 , wherein classifying the movement of each of the lidar points further comprises:
classifying a movement of the respective voxels within the three-dimensional space; and
classifying the movement of the lidar point according to a classification of the movement of the voxel mapped to the lidar point.
6. The method of claim 5 , wherein classifying the movement of the respective voxels further comprises:
for respective voxels, measuring an occupancy duration of the voxel over time by at least one lidar point of the lidar point cloud; and
comparing the occupancy duration of the voxel with an occupancy time variance threshold.
7. The method of claim 6 wherein:
the lidar point cloud has a time axis; and
the instructions are further configured to cause the one or more processors to track the movement of the respective objects over the time axis of the lidar point cloud.
8. The method of claim 6 further comprising:
measuring the occupancy duration of the voxel further comprising: measuring a standard deviation of the occupancy duration of the voxel; and
comparing the occupancy duration of the voxel further comprising: comparing the standard deviation of the occupancy duration of the voxel with a standard deviation threshold.
9. The method of claim 6 further comprising classifying the movement of each identified object including invoking the movement classifier to select the classification of the movement of the object.
10. The method of claim 1 , the instructions further configured to, after classifying the movement of each identified object, map the lidar points of the lidar point cloud to the respective objects.
11. The method of claim 1 wherein:
each lidar point cloud having lidar points associated with one of the identified objects has a time axis; and
the instructions further configured to cause the one or more processors to, for each identified object, estimate at least one vector of the object over the time axis of the lidar point cloud.
12. The method of claim 11 , the instructions further configured to cause the one or more processors to, for respective lidar points associated with the object, subtract the at least one vector of the object from the lidar point.
13. The method of claim 11 further comprising:
the lidar point clouds being detected by a vehicle having at least one vehicle vector; and
estimating the at least one vector of the object further comprising: subtracting the at least one vehicle vector of the vehicle from the at least one vector of the object.
14. The method of claim 1 , the instructions further configured to cause the one or more processors to, for each identified object, estimate a position and an orientation of the object according to the lidar points of the lidar point cloud that are associated with the object.
15. The method of claim 1 , the instructions further configured to classify each object as an object type according to the movement of the object.
16. The method of claim 15 further comprising:
an object type classifier trained to select an object type classification of each identified object; and
classifying the object type of each identified object includes invoking the object type classifier to select the object type classification of the object.
17. The method of claim 1 wherein:
the lidar point clouds are associated with at least one image; and
the instructions are further configured to cause the one or more processors to, for respective objects classified as a moving object, remove the moving object from the at least one image.
18. A system for detecting movement of objects depicted in a lidar point cloud on a device comprising one or more processors, the system comprising:
an object identifier configured to:
receive, from a lidar detector, at least a plurality and a second plurality of reflected lidar points, each plurality of lidar points forming a lidar point cloud;
map of the lidar points in each lidar point cloud to a voxel in a three-dimensional voxel space;
identify at least one object represented by a voxel cluster of voxels sharing an adjacency within the three-dimensional voxel space, wherein adjacency of two voxels is determined by a specified number of voxels between said two voxels;
for each of the plurality of lidar points in each lidar point cloud:
associate the lidar point with the identified object, and
classify the movement of the lidar point according to the identified object; and
for each identified object, classify the movement of the object according to the movement of the respective lidar points associated with the object.
19. A computer-readable storage medium comprising instructions that, when executed on one or more processors of a device, cause the device to detect movement of objects depicted in a lidar point cloud by:
executing on the one or more processors, instructions configured to:
receive, from a lidar detector, at least a first plurality and a second plurality of reflected lidar points, each plurality of lidar points forming a lidar point cloud;
map the lidar points in each lidar point cloud to a voxel in a three-dimensional voxel space;
identify at least one object represented by a voxel cluster of voxels sharing an adjacency within the three-dimensional voxel space, wherein adjacency of two voxels is determined by a specified number of voxels between said two voxels;
for each of the plurality of lidar points in each lidar point cloud:
associate the lidar point with the identified object, and
classify the movement of the lidar point according to the identified object; and
for each identified object, classify the movement of the object according to the movement of the respective lidar points associated with the object.