System for determining low-height obstacle for autonomous mobile device movement

1 . An autonomous mobile device (AMD) comprising:

a depth image sensor;

one or more motors;

one or more processors; and

one or more computer readable media storing processor executable instructions which, when executed using the one or more processors, cause the AMD to:

acquire depth image data of a physical space using the depth image sensor;

determine segmentation data based on the depth image data, wherein the segmentation data is indicative of, for at least a portion of pixels within the depth image data:

a first class indicative of a floor, or

a second class indicative of a class other than the floor;

determine, based on the depth image data, point cloud data indicative of coordinates of a plurality of points with respect to the physical space;

determine, based on the point cloud data that is associated with the first class, floor plane data indicating an estimated floor plane that is indicative of a floor that the AMD is supported by;

determine a first cluster based on the segmentation data, wherein the first cluster comprises pixels associated with the second class;

determine, based on the first cluster, a first bounding box that encompasses the first cluster;

determine, for the first bounding box, a first metric based on the point cloud data and the estimated floor plane, wherein the first metric is determined by:

determining, based on the first bounding box and the point cloud data, a first set of the plurality of points that are associated with pixels within the first bounding box,

determining, based on the point cloud data and the estimated floor plane, a first count of the points in the first set that are less than a first threshold height above the estimated floor plane, and

determining, based on the point cloud data and the estimated floor plane, a second count of the points of the first set that are less than a second threshold height above the estimated floor plane,

wherein the second threshold height is greater than the first threshold height, and

wherein the first metric is determined based on the first count of the points and the second count of the points;

determine that the first metric is less than a first threshold;

determine obstacle data that indicates the first cluster is a low-height obstacle (LHO); and

operate the one or more motors to move the AMD within the physical space, based at least in part on the obstacle data.

2 . The AMD of claim 1 , wherein the one or more computer readable media store processor executable instructions which, when executed using the one or more processors, cause the AMD to:

determine a specified region comprising pixels of the depth image data that are between a horizon boundary line that is associated with the physical space as presented in the depth image data and a lower boundary line that is below the horizon boundary line, with respect to local vertical; and

wherein:

the point cloud data is determined based on the pixels within the specified region; and

the first cluster is within the specified region.

3 . The AMD of claim 1 , wherein the one or more computer readable media store processor executable instructions which, when executed using the one or more processors, cause the AMD to:

determine a first set of pixels within the segmentation data that are:

associated with the second class,

contiguous, and

either bounded by an edge of the segmentation data or a pixel that is associated with the first class; and

wherein the first cluster comprises the first set of pixels.

4 . The AMD of claim 1 , wherein the one or more computer readable media store processor executable instructions which, when executed using the one or more processors, cause the AMD to:

determine, based on the first cluster, a second bounding box that encompasses the first cluster;

determine a first area of the second bounding box;

determine that the first area is greater than a threshold area; and

divide the second bounding box into the first bounding box and one or more additional bounding boxes.

5 . The AMD of claim 1 , wherein

the first metric is determined by dividing the first count of the points by the second count of the points.

6 . The AMD of claim 1 , wherein the one or more computer readable media store processor executable instructions which, when executed using the one or more processors, cause the AMD to:

determine path plan data indicative of a path through the physical space that passes at least through a portion of the physical space that is associated with the LHO indicated by the obstacle data; and

wherein the one or more motors are operated to move the AMD along the path and through the portion of the physical space that is associated with the LHO.

7 . A computer-implemented method comprising:

acquiring distance data of a physical space;

determining segmentation data based on the distance data;

determining a first cluster based on the segmentation data, wherein the first cluster comprises pixels associated with a first class;

determining, based on the first cluster, a first bounding box that encompasses the first cluster;

determining, based on the distance data, a first set of height data indicative of a height above a floor plane of a plurality of points associated with the first bounding box;

determining, for the first bounding box, a first metric based on the first set of height data, wherein the determining the first metric comprises:

determining, based on the first bounding box and the first set of height data, a first set of heights that are associated with pixels within the first bounding box,

determining, based on the first set of heights, a first count of the points that are less than a first threshold height, and

determining, based on the first set of heights, a second count of the points that are less than a second threshold height,

wherein the first metric is determined based on the first count of the points and the second count of the points;

determining, based on the first metric, obstacle data that indicates a portion of the physical space that is associated with an obstacle; and

operating, based on the obstacle data, one or more motors to move an autonomous mobile device within the physical space.

8 . The method of claim 7 , further comprising:

determining a specified region comprising pixels of the distance data that are between:

a) a horizon boundary line that is associated with the physical space as presented in the distance data, and

b) a lower boundary line that is below the horizon boundary line, with respect to local vertical; and

wherein:

the first set of height data is determined based on the pixels within the specified region; and

the first cluster is within the specified region.

9 . The method of claim 7 , further comprising:

determining, based on the first cluster, a second bounding box that encompasses the first cluster;

determining a first area of the second bounding box;

determining that the first area is greater than a threshold area; and

dividing the second bounding box into the first bounding box and one or more additional bounding boxes.

10 . The method of claim 7 , wherein:

the first metric is determined by dividing the first count of the points by the second count of the points.

11 . The method of claim 7 , further comprising:

determining, based on the first set of height data, that the obstacle is a low-height obstacle;

determining path plan data indicative of a path through the physical space that passes through at least a portion of the physical space that is associated with the obstacle indicated by the obstacle data; and

wherein the one or more motors are operated to move the autonomous mobile device along the path and through the portion of the physical space that is associated with the obstacle.

12 . The method of claim 7 , further comprising:

determining, based on the distance data, point cloud data indicative of coordinates of the plurality of points with respect to the physical space;

determining floor plane data, based on the point cloud data, that is indicative of an estimated floor plane the autonomous mobile device is supported by; and

wherein the first set of height data is determined based on the coordinates of the plurality of points and the floor plane data.

13 . A mobile robotic device comprising:

one or more sensors;

one or more motors;

one or more processors; and

one or more computer readable media storing processor executable instructions which, when executed using the one or more processors, cause the mobile robotic device to:

determine, using the one or more sensors, first data comprising;

distance data indicating a set of distances for pixel locations for one or more frames, and

image data indicating a set of pixel values for pixel locations for the one or more frames;

determine, based on the image data and using a machine learning model, a first set of one or more pixels associated with a potential detected obstacle;

determine, based on the first data, point cloud data;

determine, based on the point cloud data, height data indicative of a height above a floor plane of a plurality of points associated with the first set of one or more pixels;

determine, using the height data,

a first number of points associated with the first set that have a height value below a first threshold, and

a second number of points associated with the first set that have a height value below a second threshold;

determine, based on the first number of points and the second number of points, obstacle data that indicates a portion of a physical space that is associated with the first set is an obstacle; and

operate, based on the obstacle data, the one or more motors to move the mobile robotic device within physical space.

14 . The mobile robotic device of claim 13 , wherein the obstacle data indicates that the portion of the physical space that is associated with the first set is a low-height obstacle.

15 . The mobile robotic device of claim 13 , wherein the obstacle data indicates that the portion of the physical space that is associated with the first set is an obstacle that is not a low-height obstacle.

16 . The mobile robotic device of claim 13 , wherein the image data comprises grayscale image data determined based on depth data.

17 . The mobile robotic device of claim 13 , wherein the one or more computer readable media store processor executable instructions which, when executed using the one or more processors, cause the mobile robotic device to:

determine, based on the image data and using the machine learning model, segmentation data indicating one or more segmentation regions, including a first segmentation region; and

determine, based on the image data and using the machine learning model, a first class for the first segmentation region;

wherein the first set of pixels is determined based on the segmentation data.

18 . The mobile robotic device of claim 13 , wherein the one or more computer readable media store processor executable instructions which, when executed using the one or more processors, cause the mobile robotic device to:

determine, based on the image data, a first bounding box associated with the potential detected obstacle;

wherein the first set of pixels is determined based on the first bounding box.

19 . The mobile robotic device of claim 13 , wherein the one or more computer readable media store processor executable instructions which, when executed using the one or more processors, cause the mobile robotic device to:

determine, based on the image data and using the machine learning model, second data indicating a first detected object;

wherein the first set of pixels is determined based on the second data.

20 . The mobile robotic device of claim 13 , wherein, in determining the first number of points, points having a negative height value qualify as below the first threshold.