Mobile robot and a method for controlling the mobile robot
A mobile robot and a method for controlling a mobile robot are provided. The method includes: accessing a 3D map representation of an environment including a respective indication of a crosswalk, a waiting area, and a visual traffic indicator; detecting an object located in the waiting area for generating a bounding box of the object; mapping a position of the bounding box to the 3D map representation; determining a visual occlusion zone of the object in the waiting area; determining a reduced waiting area as a difference between the waiting area and the visual occlusion area, and triggering control of the propulsion system for moving the mobile robot to the reduced waiting area.
1. A mobile robot having a Light Detection and Ranging (LIDAR) sensor, a camera sensor having a field of view, and a propulsion system for propelling the mobile robot in an environment, a processor being communicatively coupled with the LIDAR sensor, the camera sensor, and the propulsion system, the processor being configured to:
access a 3D map representation of the environment including a respective indication of a crosswalk, a waiting area for the crosswalk, and a visual traffic indicator for the crosswalk,
the 3D map representation indicative of a position of the visual traffic indicator and its orientation relative to the waiting area;
detect an object located in the waiting area by using at least one of camera data from the camera sensor and LIDAR data from the LIDAR sensor, thereby generating a bounding box of the object;
map a position of the bounding box to the 3D map representation;
determine a visual occlusion zone of the object in the waiting area using the position of the bounding box and the orientation of the visual traffic indicator relative to the waiting area,
the visual occlusion zone being an estimated portion of the waiting area in which the visual traffic indicator is occluded from the field of view of the camera sensor by the object if the mobile robot is located in the estimated portion;
determine a reduced waiting area as a difference between the waiting area and the visual occlusion area,
the visual traffic indicator being visible in the field of view of the camera sensor if the mobile robot is located in the reduced waiting area; and
trigger control of the propulsion system for moving the mobile robot to the reduced waiting area.
2. The mobile robot of claim 1 , wherein the processor is further configured to:
compare the reduced waiting area to a pre-determined threshold area; and
in response to the reduced waiting area being above the pre-determined threshold area, control the propulsion system for moving the mobile robot to the reduced waiting area;
in response to the reduced waiting area being below the pre-determined threshold area:
determine a shell of the object using the LIDAR data of the LIDAR sensor, the shell being a more accurate estimation of a shape of the object than the bounding box;
map a position of the shell on the 3D representation;
determine an other visual occlusion zone of the object in the waiting area using the position of the shell and the orientation of the visual indicator relative to the waiting area,
the other visual occlusion zone being an other estimated portion of the waiting area in which the visual traffic indicator will be occluded from the field of view of the camera sensor by the object if the mobile robot is located in the other estimated portion,
the other estimated portion being smaller than the estimated portion;
determine an other reduced waiting area as a difference between the waiting area and the other visual occlusion area,
the visual traffic indicator being visible in the field of view of the camera sensor is the mobile robot is located in the other reduced waiting area;
compare the other reduced waiting area to the pre-determined threshold area; and
in response to the other reduced waiting area being above the pre-determined threshold area:
trigger control the propulsion system for moving the mobile robot to the other reduced waiting area.
3. The mobile robot of claim 1 , wherein the processor is further configured to locally store at least a portion of the 3D map representation.
4. The mobile robot of claim 1 , wherein to detect, the processor is further configured to employ an Object Detection Neural Network (ODNN) onto the at least one of the camera data and the LIDAR data.
5. The mobile robot of claim 1 , wherein to determine the visual occlusion zone the processor is further configured to project at least one corner of the bounding box onto the waiting area by casting a ray in the 3D map representation from the visual traffic indicator onto the waiting area and such that the ray is passes through the at least one of the corners of the bounding box.
6. The mobile robot of claim 5 , wherein the ray is more than one ray, the processor being configured to cast the more than one rays from the visual traffic indicator towards the bounding box to define a shadow projection of the bounding box in the waiting area, the shadow projection of the bounding box representing the visual occlusion zone.
7. The mobile robot of claim 1 , wherein the camera sensor is located at a pre-determined distance above a terrain on which the mobile robot moves, and wherein the processor is further configured to use the pre-determined distance to determine the visual occlusion zone, the visual occlusion zone being located at the pre-determined distance above the waiting area.
8. The mobile robot of claim 1 , wherein the processor is further configured to control the propulsion system for stopping the mobile robot in the reduced waiting area.
9. The mobile robot of claim 1 , wherein the processor is further configured to monitor a state of the visual traffic indicator.
10. The mobile robot of claim 1 , wherein the visual traffic indicator is a traffic light.
11. The mobile robot of claim 1 , wherein the processor is an onboard processor of the mobile robot.
12. The mobile robot of claim 1 , wherein the processor is a remote processor of a server, the server being communicatively coupled with the mobile robot.
13. A method for controlling a mobile robot in an environment, comprising:
accessing a 3D map representation of the environment including a respective indication of a crosswalk, a waiting area for the crosswalk, and a visual traffic indicator for the crosswalk,
the 3D map representation indicative of a position of the visual traffic indicator and its orientation relative to the waiting area
detecting, using at least one sensor of the mobile robot, an object in the waiting area, thereby generating a bounding box of the object:
casting, in the 3D map representation, two rays from the visual traffic indicator towards the bounding box to define a shadow projection of the bounding box in the waiting area, the shadow projection representing a visual occlusion zone of the object in the waiting area,
the visual occlusion zone being an estimated portion of the waiting area in which the visual traffic indicator is occluded from the field of view of the camera sensor by the object if the mobile robot is located in the estimated portion;
determining a reduced waiting area by subtracting the visual occlusion zone from the waiting area;
in response to the reduced waiting area being above a pre-determined threshold area, controlling the mobile robot so that the robot stops inside the reduced waiting area, the visual traffic indicator being visible in the field of view of the camera sensor when the mobile robot is located in the reduced waiting area.
14. The method of claim 13 , wherein in response to the reduced waiting area being lower than the pre-determined threshold area,
generating, using the at least one sensor, a shell for object, the shell being a more accurate estimation of a shape of the object than the bounding box;
casting, in the 3D map representation, two rays from the visual traffic indicator towards the shell to define an other shadow projection of the shell in the waiting area, the other shadow projection representing an other visual occlusion zone of the object in the waiting area,
the other visual occlusion zone being smaller than the visual occlusion zone;
determining an other reduced waiting area by subtracting the other visual occlusion zone from the waiting area;
in response to the other reduced waiting area being above the pre-determined threshold area, controlling the mobile robot so that the robot stops inside the other reduced waiting area.
15. The method of claim 13 , wherein the method further comprises:
continuously tracking a state of the visual traffic indicator using at least one camera sensor of the mobile robot.
16. A method of controlling a mobile robot having a Light Detection and Ranging (LIDAR) sensor, a camera sensor having a field of view, and a propulsion system for propelling the mobile robot in an environment, a processor being communicatively coupled with the LIDAR sensor, the camera sensor, and the propulsion system, the method being executable by the processor, the method comprising:
accessing a 3D map representation of the environment including a respective indication of a crosswalk, a waiting area for the crosswalk, and a visual traffic indicator for the crosswalk,
the 3D map representation indicative of a position of the visual traffic indicator and its orientation relative to the waiting area;
detecting an object located in the waiting area by using at least one of camera data from the camera sensor and LIDAR data from the LIDAR sensor, thereby generating a bounding box of the object;
mapping a position of the bounding box to the 3D map representation;
determining a visual occlusion zone of the object in the waiting area using the position of the bounding box and the orientation of the visual traffic indicator relative to the waiting area,
the visual occlusion zone being an estimated portion of the waiting area in which the visual traffic indicator is occluded from the field of view of the camera sensor by the object if the mobile robot is located in the estimated portion;
determining a reduced waiting area as a difference between the waiting area and the visual occlusion area,
the visual traffic indicator being visible in the field of view of the camera sensor if the mobile robot is located in the reduced waiting area; and
triggering control of the propulsion system for moving the mobile robot to the reduced waiting area.
17. The method of claim 16 , wherein the method further comprises:
comparing the reduced waiting area to a pre-determined threshold area; and
in response to the reduced waiting area being above the pre-determined threshold area, controlling the propulsion system for moving the mobile robot to the reduced waiting area;
in response to the reduced waiting area being below the pre-determined threshold area:
determining a shell of the object using the LIDAR data of the LIDAR sensor, the shell being a more accurate estimation of a shape of the object than the bounding box;
mapping a position of the shell on the 3D representation;
determining an other visual occlusion zone of the object in the waiting area using the position of the shell and the orientation of the visual indicator relative to the waiting area,
the other visual occlusion zone being an other estimated portion of the waiting area in which the visual traffic indicator will be occluded from the field of view of the camera sensor by the object if the mobile robot is located in the other estimated portion,
the other estimated portion being smaller than the estimated portion;
determining an other reduced waiting area as a difference between the waiting area and the other visual occlusion area,
the visual traffic indicator being visible in the field of view of the camera sensor is the mobile robot is located in the other reduced waiting area;
comparing the other reduced waiting area to the pre-determined threshold area; and
in response to the other reduced waiting area being above the pre-determined threshold area:
triggering control the propulsion system for moving the mobile robot to the other reduced waiting area.
18. The method of claim 16 , wherein at least a portion of the 3D map representation is locally stored.
19. The method of claim 16 , wherein the detecting includes employing an Object Detection Neural Network (ODNN) onto the at least one of the camera data and the LIDAR data.
20. The method of claim 16 , wherein the determining the visual occlusion zone includes projecting at least one corner of the bounding box onto the waiting area by casting a ray in the 3D map representation from the visual traffic indicator onto the waiting area and such that the ray is passes through the at least one of the corners of the bounding box.