Multi-sensor environmental mapping

1. A method for controlling an unmanned aerial vehicle (UAV), the method comprising:

receiving first sensor data relative to a first coordinate system from a first sensor, the first sensor data comprising first obstacle occupancy information indicative of relative locations of a first set of obstacles in reference to the UAV in the first coordinate system, wherein the first sensor is selected from a plurality of sensors carried by the UAV;

receiving second sensor data relative to a second coordinate system from a second sensor, the second sensor data comprising second obstacle occupancy information indicative of relative locations of a second set of obstacles in reference to the UAV in the second coordinate system, wherein the second sensor is selected from the plurality of sensors and is different from the first sensor, and wherein the first set of obstacles and the second set of obstacles have at least a subset of obstacles in common;

generating, by converting the first sensor data and the second sensor data into a single coordinate system using sensor calibration data, an obstacle occupancy grid map based on the first obstacle occupancy information and the second obstacle occupancy information, the obstacle occupancy grid map comprising a plurality of obstructed spaces and a plurality of unobstructed spaces; and

effecting the UAV to navigate using the obstacle occupancy grid map to perform obstacle avoidance based on the plurality of obstructed spaces and the plurality of unobstructed spaces.

2. The method of claim 1 , wherein converting the first sensor data and the second sensor data into the single coordinate system using the sensor calibration data, further comprises:

obtaining first coordinates of the subset of obstacles in common from the first sensor data based on the first obstacle occupancy information;

obtaining second coordinates of the subset of obstacles in common from the second sensor data based on the second obstacle occupancy information;

computing the sensor calibration data comprising sensor parameters indicative of a spatial relationship between the first sensor and the second sensor, the sensor parameters comprising a rotation matrix and a transform matrix for converting between the first coordinate system and the second coordinate system, wherein the rotation matrix and the transform matrix are computed based on the first coordinates and the second coordinates of the subset of obstacles in common; and

converting the first sensor data and the second sensor data into the single coordinate system using the rotation matrix and the transform matrix.

3. The method of claim 2 , wherein converting the first sensor data and the second sensor data into the single coordinate system further comprises using an internal matrix or a scale factor, wherein the internal matrix is associated with intrinsic parameters calculated in prior sensor calibration, and wherein the scale factor is calculated based at least on coordinates of an obstacle within the subset of obstacles in common along a single sensing direction.

4. The method of claim 1 , further comprising:

determining a current location of the UAV using at least one of the plurality of sensors;

receiving, from a remote controller or a mobile device, instructions to autonomously return to an initial location or fly to a specified location; and

determining a flight path among a plurality of potential flight paths from the current location to the initial location or the specified location based on a set of operational criteria.

5. The method of claim 4 , further comprising:

detecting obstacles along the flight path using the obstacle occupancy grid map;

modifying the flight path based on the detected obstacles to avoid navigating within the obstructed spaces of the obstacle occupancy grid map; and

effecting the UAV to move along the modified flight path to autonomously return to the initial location or fly to the specified location.

6. The method of claim 5 , wherein modifying the flight path based on the detected obstacles further comprises:

for each detected obstacle along the flight path, calculating confidence information based on a collision risk, wherein the collision risk is associated with a minimum distance between the detected obstacle and the flight path; and

modifying the flight path based on the confidence information.

7. The method of claim 4 , wherein the set of operational criteria is associated with minimizing at least one of a flight time, a flight distance, an energy consumption, a number of obstacles along each flight path of the plurality of potential flight paths, distances of the UAV from obstacles along each flight path of the plurality of potential flight paths, or an altitude range of the UAV.

8. The method of claim 1 , wherein the plurality of sensors are selected from: a vision sensor, an image sensor, a camera, a proximity sensor, a range sensor, a light sensor, a LiDAR sensor, an acoustic sensor, an ultrasonic sensor, a location sensor, a Global Positioning System (GPS) sensor, a speed sensor, a pressure sensor, a thermal sensor, and an environmental sensor.

9. The method of claim 1 , wherein the single coordinate system is one of a local coordinate system, a global coordinate system, a world coordinate system, or a body coordinate system defined relative to the UAV.

10. A system for controlling an unmanned aerial vehicle (UAV), the system comprising:

a plurality of sensors carried by the UAV; and

one or more processors, individually or collectively configured to:

receive first sensor data relative to a first coordinate system from a first sensor of the plurality of sensors, the first sensor data comprising first obstacle occupancy information indicative of relative locations of a first set of obstacles in reference to the UAV in the first coordinate system;

receive second sensor data relative to a second coordinate system from a second sensor of the plurality of sensors, the second sensor data comprising second obstacle occupancy information indicative of relative locations of a second set of obstacles in reference to the UAV in the second coordinate system, wherein the second sensor is different from the first sensor, and wherein the first set of obstacles and the second set of obstacles have at least a subset of obstacles in common;

generate, by converting the first sensor data and the second sensor data into a single coordinate system using sensor calibration data, an obstacle occupancy grid map based on the first obstacle occupancy information and the second obstacle occupancy information, the obstacle occupancy grid map comprising a plurality of obstructed spaces and a plurality of unobstructed spaces; and

effect the UAV to navigate using the obstacle occupancy grid map to perform obstacle avoidance based on the plurality of obstructed spaces and the plurality of unobstructed spaces.

11. The system of claim 10 , wherein to convert the first sensor data and the second sensor data into the single coordinate system using the sensor calibration data, the one or more processors are, individually or collectively further configured to:

obtain first coordinates of the subset of obstacles in common from the first sensor data based on the first obstacle occupancy information;

obtain second coordinates of the subset of obstacles in common from the second sensor data based on the second obstacle occupancy information;

compute the sensor calibration data comprising sensor parameters indicative of a spatial relationship between the first sensor and the second sensor, the sensor parameters comprising a rotation matrix and a transform matrix for converting between the first coordinate system and the second coordinate system, wherein the rotation matrix and the transform matrix are computed based on the first coordinates and the second coordinates of the subset of obstacles in common; and

convert the first sensor data and the second sensor data into the single coordinate system using the rotation matrix and the transform matrix, wherein the single coordinate system is one of a local coordinate system, a global coordinate system, a world coordinate system, or a body coordinate system defined relative to the UAV.

12. The system of claim 11 , wherein to convert the first sensor data and the second sensor data into the single coordinate system, the one or more processors are, individually or collectively further configured to use an internal matrix or a scale factor, wherein the internal matrix is associated with intrinsic parameters calculated in prior sensor calibration, and wherein the scale factor is calculated based at least on coordinates of an obstacle within the subset of obstacles in common along a single sensing direction.

13. The system of claim 10 , wherein the one or more processors are, individually or collectively further configured to:

determine a current location of the UAV using at least one of the plurality of sensors;

receive, from a remote controller or a mobile device, instructions to autonomously return to an initial location or fly to a specified location;

determine a flight path among a plurality of potential flight paths from the current location to the initial location or the specified location based on a set of operational criteria, wherein the set of operational criteria is associated with minimizing at least one of a flight time, a flight distance, an energy consumption, a number of obstacles along each flight path of the plurality of potential flight paths, distances of the UAV from obstacles along each flight path of the plurality of potential flight paths, or an altitude range of the UAV;

detect obstacles along the flight path using the obstacle occupancy grid map;

modify the flight path based on the detected obstacles to avoid the obstructed spaces of the obstacle occupancy grid map; and

effect the UAV to move along the modified flight path to autonomously return to the initial location or fly to the specified location.

14. The system of claim 13 , wherein to modify the flight path based on the detected obstacles, the one or more processors are, individually or collectively further configured to:

for each detected obstacle along the flight path, calculate confidence information based on a collision risk, wherein the collision risk is associated with a minimum distance between the detected obstacle and the flight path; and

modify the flight path based on the confidence information.

15. The system of claim 10 , wherein the plurality of sensors are selected from: a vision sensor, an image sensor, a camera, a proximity sensor, a range sensor, a light sensor, a LiDAR sensor, an acoustic sensor, an ultrasonic sensor, a location sensor, a Global Positioning System (GPS) sensor, a speed sensor, a pressure sensor, a thermal sensor, and an environmental sensor.

16. An unmanned aerial vehicle (UAV), comprising:

a plurality of sensors carried by the UAV;

a propulsion system; and

one or more processors, individually or collectively configured to:

receive first sensor data relative to a first coordinate system from a first sensor of the plurality of sensors, the first sensor data comprising first obstacle occupancy information indicative of relative locations of a first set of obstacles in reference to the UAV in the first coordinate system;

receive second sensor data relative to a second coordinate system from a second sensor of the plurality of sensors, the second sensor data comprising second obstacle occupancy information indicative of relative locations of a second set of obstacles in reference to the UAV in the second coordinate system, wherein the second sensor is different from the first sensor, and wherein the first set of obstacles and the second set of obstacles have at least a subset of obstacles in common;

generate, by converting the first sensor data and the second sensor data into a single coordinate system using sensor calibration data, an obstacle occupancy grid map based on the first obstacle occupancy information and the second obstacle occupancy information, the obstacle occupancy grid map comprising a plurality of obstructed spaces and a plurality of unobstructed spaces; and

control the propulsion system for effecting the UAV to navigate using the obstacle occupancy grid map to perform obstacle avoidance based on the plurality of obstructed spaces and the plurality of unobstructed spaces.

17. The UAV of claim 16 , wherein to convert the first sensor data and the second sensor data into the single coordinate system using the sensor calibration data, the one or more processors are, individually or collectively further configured to:

obtain first coordinates of the subset of obstacles in common from the first sensor data based on the first obstacle occupancy information;

obtain second coordinates of the subset of obstacles in common from the second sensor data based on the second obstacle occupancy information;

compute the sensor calibration data comprising sensor parameters indicative of a spatial relationship between the first sensor and the second sensor, the sensor parameters comprising a rotation matrix and a transform matrix for converting between the first coordinate system and the second coordinate system, wherein the rotation matrix and the transform matrix are computed based on the first coordinates and the second coordinates of the subset of obstacles in common; and

convert the first sensor data and the second sensor data into the single coordinate system using the rotation matrix and the transform matrix, wherein the single coordinate system is one of a local coordinate system, a global coordinate system, a world coordinate system, or a body coordinate system defined relative to the UAV.

18. The UAV of claim 17 , wherein to convert the first sensor data and the second sensor data into the single coordinate system, the one or more processors are, individually or collectively further configured to use an internal matrix or a scale factor, wherein the internal matrix is associated with intrinsic parameters calculated in prior sensor calibration, and wherein the scale factor is calculated based at least on coordinates of an obstacle within the subset of the obstacles in common along a single sensing direction.

19. The UAV of claim 16 , wherein the one or more processors are, individually or collectively further configured to:

determine a current location of the UAV using at least one of the plurality of sensors;

receive, from a remote controller or a mobile device, instructions to autonomously return to an initial location or fly to a specified location;

determine a flight path among a plurality of potential flight paths from the current location to the initial location or the specified location based on a set of operational criteria, wherein the set of operational criteria is associated with minimizing at least one of a flight time, a flight distance, an energy consumption, a number of obstacles along each flight path of the plurality of potential flight paths, distances of the UAV from obstacles along each flight path of the plurality of potential flight paths, or an altitude range of the UAV;

detect obstacles along the flight path using the obstacle occupancy grid map;

modify the flight path based on the detected obstacles to avoid the obstructed spaces of the obstacle occupancy grid map; and

control the propulsion system for effecting the UAV to move along the modified flight path to autonomously return to the initial location or fly to the specified location.

20. The UAV of claim 19 , wherein to modify the flight path based on the detected obstacles, the one or more processors are, individually or collectively further configured to:

for each detected obstacle along the flight path, calculate confidence information based on a collision risk, wherein the collision risk is associated with a minimum distance between the detected obstacle and the flight path; and

modify the flight path based on the confidence information.

21. The UAV of claim 16 , wherein the plurality of sensors are selected from: a vision sensor, an image sensor, a camera, a proximity sensor, a range sensor, a light sensor, a LiDAR sensor, an acoustic sensor, an ultrasonic sensor, a location sensor, a Global Positioning System (GPS) sensor, a speed sensor, a pressure sensor, a thermal sensor, and an environmental sensor.