Autonomous machine navigation with object detection and 3D point cloud
Autonomous machine navigation techniques may determine vision-based pose data based on feature data and object recognition data extracted from images. The vision-based pose data may be used to generate a three-dimensional point cloud that represents at least a work region. The vision-based pose data may be used to determine an operational vision-based pose relative to the three-dimensional point cloud.
1. A method for navigation of an autonomous machine in a work region, the method comprising:
generating one or more images of at least the work region using one or more cameras of the autonomous machine;
generating object recognition data indicative of any objects detected in the one or more images;
in response to the object recognition data indicating at least one object detected in the one or more images, determining at least one object area corresponding to the at least one object or determining object descriptor data that indicates a characteristic of the at least one object;
generating feature data extracted from the one or more images by prioritizing features to match based on the object descriptor data;
generating vision-based pose data based on the object recognition data and the feature data; and
directing navigation of the autonomous machine based on the vision-based pose data.
2. The method according to claim 1 , further comprising:
generating a three-dimensional point cloud (3DPC) that represents the features in the work region based on the vision-based pose data, wherein directing navigation of the autonomous machine is further based on the 3DPC.
3. The method according to claim 2 , further comprising generating operational vision-based pose data based on feature matching the 3DPC and operational feature data at least partially based on the object descriptor data.
4. The method according to claim 2 , further comprising generating a visual representation of the 3DPC and any objects in the work region based on the feature data comprising the object descriptor data.
5. The method according to claim 2 , wherein directing navigation of the autonomous machine comprises:
generating one or more operational images of at least the work region using the one or more cameras of the autonomous machine;
generating operational object recognition data indicative of any objects detected in the one or more operational images;
generating operational feature data extracted from the one or more operational images, wherein the operational feature data excludes any features associated with any objects or comprises operational object descriptor data corresponding to any objects; and
generating operational vision-based pose data based on the operational feature data, wherein directing navigation of the autonomous machine is further based on the operational vision-based pose data.
6. The method according to claim 2 , wherein directing navigation of the autonomous machine comprises:
generating one or more operational images of at least the work region using the one or more cameras of the autonomous machine;
generating operational feature data extracted from the one or more operational images; and
generating operational vision-based pose data based on the operational feature data, wherein directing navigation of the autonomous machine is further based on the operational vision-based pose data.
7. The method according to claim 1 , wherein directing navigation of the autonomous machine comprises:
generating one or more operational images of at least the work region using the one or more cameras of the autonomous machine;
generating operational object recognition data indicative of any objects detected in the one or more operational images;
generating operational feature data extracted from the one or more operational images, wherein the operational feature data excludes any features associated with any objects or comprises operational object descriptor data corresponding to any objects; and
generating operational vision-based pose data based on the operational feature data, wherein directing navigation of the autonomous machine is further based on the operational vision-based pose data.
8. The method according to claim 1 , wherein directing navigation of the autonomous machine comprises:
generating one or more operational images of at least the work region using the one or more cameras of the autonomous machine;
generating operational feature data extracted from the one or more operational images; and
generating operational vision-based pose data based on the operational feature data, wherein directing navigation of the autonomous machine is further based on the operational vision-based pose data.
9. The method according to claim 1 , further comprising:
in response to the object recognition data indicating the at least one object detected in the one or more images, determining the object descriptor data corresponding to the at least one object, wherein the feature data comprises the object descriptor data.
10. The method according to claim 1 , wherein generating the one or more images comprises generating one or more operational images during an online mode of the autonomous machine.
11. The method according to claim 1 , wherein the at least one object comprises a temporary object, wherein a change in appearance or location of the temporary object affects feature extraction.
12. The method according to claim 1 , wherein the object recognition data is determined at least partially based on user input data.
13. The method according to claim 1 , wherein prioritizing the features to match comprises starting with features associated more permanent or reliable objects before feature matching other features associated with more temporary or less reliable objects.
14. An autonomous machine adapted to carry out the method of claim 1 , the autonomous machine comprising:
a housing coupled to a maintenance implement;
a set of wheels supporting the housing over a ground surface;
a propulsion controller operably coupled to the set of wheels;
a vision system comprising the one or more cameras adapted to capture image data; and
a navigation system operably coupled to the vision system and the propulsion controller, the navigation system adapted to direct navigation of the autonomous machine within the work region.
15. A method for navigation of an autonomous machine in a work region, the method comprising:
generating one or more images of at least the work region using one or more cameras of the autonomous machine;
generating object recognition data indicative of any objects detected in the one or more images;
in response to the object recognition data indicating at least one object detected in the one or more images, determining at least one object area corresponding to the at least one object or determining object descriptor data that indicates a characteristic of the at least one object;
generating feature data extracted from the one or more images excluding the at least one object area or generating the feature data by prioritizing features to match based on the object descriptor data;
generating vision-based pose data based on the object recognition data and the feature data;
directing navigation of the autonomous machine based on the vision-based pose data;
generating a three-dimensional point cloud (3DPC) that represents the features in the work region based on the vision-based pose data, wherein directing navigation of the autonomous machine is further based on the 3DPC; and
generating operational vision-based pose data based on feature matching the 3DPC and operational feature data at least partially based on the object descriptor data, wherein the feature matching the 3DPC and the operational feature data comprises one of more of:
feature matching using extracted features corresponding to same objects or a same classification of objects based on the object descriptor data;
feature matching using the object descriptor data to update a matching probability before comparing the matching probability to a matching threshold;
feature matching using prioritization of the features to match based on the object descriptor data; and
feature matching using object descriptor data to exclude features in the 3DPC obstructed from view at an estimated operational pose.
16. The method according to claim 15 , wherein feature matching using extracted features corresponding to the same objects or the same classification of objects based on the object descriptor data further comprises only comparing features to an instance of the same objects or between instances of the same classification of objects.
17. The method according to claim 15 , wherein feature matching using the object descriptor data to update the matching probability before comparing the matching probability to the matching threshold comprises one or more of:
assigning a lower probability to first matched features with unmatched object descriptor data; and
assigning a higher probability to second matched features with lower correlation but matching object descriptor data.
18. A method for navigation of an autonomous machine in a work region, the method comprising:
generating one or more images of at least the work region using one or more cameras of the autonomous machine;
generating object recognition data indicative of any objects detected in the one or more images;
in response to the object recognition data indicating at least one object detected in the one or more images, determining at least one object area corresponding to the at least one object or determining object descriptor data that indicates a characteristic of the at least one object;
generating feature data extracted from the one or more images excluding the at least one object area or generating the feature data by prioritizing features to match based on the object descriptor data;
generating vision-based pose data based on the object recognition data and the feature data;
generating a three-dimensional point cloud (3DPC) that represents the features in the work region based on the vision-based pose data, wherein directing navigation of the autonomous machine is further based on the 3DPC, wherein the at least one object obstructs a line of sight between an estimated operational pose and particular features, and the at least one object area is used to exclude the particular features obstructed from view to constrain points in the 3DPC for feature matching; and
directing navigation of the autonomous machine based on the vision-based pose data.
19. The method according to claim 18 , wherein the excluding of the at least one object area improves the determination of operational vision-based pose data.