Fully automated multimodal system architecture for semantic segmentation of large-scale 3D outdoor point cloud data

1. A method for semantic segmentation of 3D point cloud data, comprising:

determining ground data points of the 3D point cloud data;

categorizing non-ground data points relative to a ground surface determined from the ground data points to determine legitimate non-ground data points;

segmenting the determined legitimate non-ground and ground data points based on a set of common features by applying a machine learning algorithm trained using logical rules to segment non-ground and ground data points based on a set of common features, wherein at least some of the common features are defined by the logical rules;

determining a data structure defining a spatial relationship of the classified, segmented non-ground and ground data points by applying a graph convolutional neural network;

applying logical rules to the data structure of the features built on the segmented determined non-ground and ground data points based on their spatial relationships and incorporated within a machine learning system; and

constructing a 3D semantics model from the application of the logical rules to the data structure.

2. The method of claim 1 , wherein legitimate non-ground data points comprise non-outlier data points above a ground level.

3. The method of claim 1 , wherein determining legitimate non-ground data points, comprises:

down-sampling the 3D point cloud data to a spatially uniform resolution;

performing a connected component analysis on the raw 3Dpoint cloud data;

ordering resulting connected components from largest to smallest according to a number of points contained; and

fusing a largest number of connected components into scene points.

4. The method of claim 1 , wherein the segmenting comprises:

grouping the determined legitimate non-ground and ground data points into superpoints including local groupings of 3D points that are homogenous in terms of the common features.

5. The method of claim 1 , wherein the common features comprise at least one of color features, appearance features, geometric relationships and temporal relationships.

6. The method of claim 1 , wherein applying the logical rules comprises applying the logical rules as a declarative, first-order logic statements.

7. The method of claim 1 , wherein applying the logical rules comprises determining a loss function on a frame by frame basis to determine if the data points comply with the logical rules.

8. The method of claim 1 , further comprising:

applying a mesh compression algorithm to data points of the data structure to isolate specific segmented data points having a common feature.

9. A non-transitory machine-readable medium having stored thereon at least one program, the at least one program including instructions which, when executed by a processor, cause the processor to perform a method in a processor based system for semantic segmentation of 3D point cloud data, comprising:

determining ground data points of the 3D point cloud data;

categorizing non-ground data points relative to the ground surface to determine legitimate non-ground data points;

segmenting the determined legitimate non-ground and ground data points based on a set of common features by applying a machine learning algorithm trained using logical rules to segment non-ground and ground data points based on a set of common features, wherein at least some of the common features are defined by the logical rules;

determining a data structure defining a spatial relationship of the classified, segmented non-ground and ground data points by applying a graph convolutional neural network;

applying logical rules to the data structure of the features built on the segmented determined non-ground and ground data points based on their spatial relationships and incorporated within a machine learning system; and

constructing a 3D semantics model from the application of the logical rules to the data structure.

10. The non-transitory machine-readable medium of claim 9 , wherein legitimate non-ground data points comprise non-outlier data points above a ground level.

11. The non-transitory machine-readable medium of claim 9 , wherein determining legitimate non-ground data points, comprises:

down-sampling the 3D point cloud data to a spatially uniform resolution,

performing a 26-connected component analysis on the raw 3Dpoint cloud data;

ordering resulting connected components from largest to smallest according to a number of points contained; and

fusing a largest number of connected components into scene points.

12. The non-transitory machine-readable medium of claim 9 , wherein determining 3D ground data points of the method comprises:

applying a rule-based binary ground versus non ground classifier algorithm to the determined legitimate non-ground data points.

13. The non-transitory machine-readable medium of claim 9 , wherein segmenting the determined non-ground and ground data points of the method comprises:

grouping the legitimate non-ground and ground data points into superpoints including local groupings of 3D points that are homogenous in terms of the common features.

14. The non-transitory machine-readable medium of claim 9 , wherein applying the logical rules comprises applying the logical rules as declarative, first-order logic statements.

15. The non-transitory machine-readable medium of claim 9 , wherein applying the logical rules comprises determining a loss function on a frame by frame basis to determine if the data points comply with the logical rules.

16. The non-transitory machine-readable medium of claim 9 , wherein the method further comprises:

applying a mesh compression algorithm to the data points of the data structure to isolate specific segmented data points having a common feature.

17. A system for semantic segmentation of 3D point cloud data, comprising:

a processor; and

a memory coupled to the processor, the memory having stored therein at least one of programs or instructions executable by the processor to configure the system to:

determine ground data points of the 3D point cloud data;

categorize the non-ground data points relative to the ground surface to determine legitimate non-ground data points;

segment the determined legitimate non-ground and ground data points based on a set of common features by applying a machine learning algorithm trained using logical rules to segment non-ground and ground data points based on a set of common features, wherein at least some of the common features are defined by the logical rules;

determining a data structure defining a spatial relationship of the classified, segmented non-ground and ground data points by applying a graph convolutional neural network;

apply logical rules to the data structure of the features built on the segmented determined non-ground and ground data points based on their spatial relationships and incorporated within a machine learning system; and

construct a 3D semantics model from the application of the logical rules to the data structure.

18. The system of claim 17 , wherein the system is configured to:

apply the logical rules as at least one declarative, first-order logic statement.

19. The system of claim 17 , wherein the system is further configured to:

apply a mesh compression algorithm to data points of the data structure to isolate specific segmented data points having a common feature.

20. The system of claim 17 , wherein the data structure comprises a graph.